ka2

ka2 is a scraper and AI-driven search bot that charts the Quebec web as a knowledge graph: entities — companies, organizations, people, creators/influencers, agencies, brands — with their attributes and typed relations such as WORKS_AT, FOUNDER_OF, REPRESENTED_BY, COLLABORATES_WITH, and SPONSORED_BY, each carrying a role and provenance. It is the light, economical edition of Groupe KA, which operates three twin bots sharing the same engine: ka2 at 800 pages per day, ka4 at 3,000, and ka6 at 9,000. This repository is ka2, in its signature lime-green theme.

A Claude Haiku 4.5 agent plans missions from the dashboard and drives the pipeline through tools — search_web, map_site, scrape_page, deep_dive, and crawl_queue. Pages are fetched by Firecrawl with Scrapfly as an anti-bot fallback, at 6 concurrent pages with a 0.6-second delay, robots.txt respected, fail-fast timeouts, per-URL retries with a dead-letter queue, and a per-domain circuit breaker after 4 failures. A Haiku extractor then turns page markdown into entities and typed relations, deduplicated by normalized name, domain, and canonical Quebec region before landing in SQLite (WAL) with full provenance.

The codebase is 98.1% Python, served by a FastAPI application: a real-time SSE dashboard with KPIs, activity feed, live settings, and mission controls, plus a structured directory offering three views — cards, sortable table, and a force-directed global graph with pan/zoom — behind type/sector/region facets, search, and CSV/JSON export. A one-command deploy script targets MacLustr cluster node M4M36, creating three resilient launchd services (com.ka2.web, com.ka2.bot, com.ka2.ngrok) that expose the app on port 8799 through an ngrok tunnel at www.ka2.bot. Public data only, aligned with Quebec's Law 25 and PIPEDA.