{"id":"7acc94cd-f27f-4ff5-8e83-680196af4731","slug":"knowbase","kind":"mcp","name":"knowbase","summary":"Git-versioned, provenance-grounded knowledge layer extracted from a codebase, served to humans and AI agents via MCP","owner":"v0ropaev","repoUrl":"https://github.com/v0ropaev/knowbase","license":"AGPL-3.0","licenseTier":"strong-copyleft","trust":"community","stars":9,"match":null,"fewStars":false,"format":"mcp","createdAt":null,"addedAt":"2026-10-14T05:50:27.900Z","logoUrl":"https://avatars.githubusercontent.com/u/144015565?u=f19f115e80955778d59d43bc94abf1d74243c0b7&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["developer-tools","memory-knowledge","databases"],"tags":["ai-agents","anti-hallucination","code-intelligence","code-understanding","dependency-graph","developer-tools","git","knowledge-base"],"runtime":"python","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-14T10:04:11.516Z","sample":false,"url":"https://conduits.dev/tools/knowbase","language":"Python","transports":["stdio"],"install":{"pypi":"knowbase"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"
\n\n\"knowbase\n\n[![CI](https://github.com/v0ropaev/knowbase/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/v0ropaev/knowbase/actions/workflows/ci.yml)\n[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](./LICENSE)\n[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue.svg)](https://www.python.org/downloads/)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![Checked with mypy](https://img.shields.io/badge/mypy-strict-blue)](https://mypy-lang.org/)\n[![GHCR image](https://img.shields.io/badge/ghcr.io-knowbase-2496ED?logo=docker&logoColor=white)](https://github.com/v0ropaev/knowbase/pkgs/container/knowbase)\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/v0ropaev/knowbase/pulls)\n\n
\n\n> A versioned, provenance-grounded **knowledge layer** over a codebase — served to humans and AI agents. Not RAG-over-code.\n\nknowbase turns a git repository into a **Knowledge Layer**: a queryable, git-versioned model of what a codebase *means* — its architecture, domain entities, API contracts, dependencies, events, and business processes — where **every fact is bound to the exact lines of code it came from** (`file:line@sha`).\n\nThe one thing that makes it different: it does not embed your code and hope. It **extracts** durable knowledge and grounds each unit in a real code span. LLMs and embeddings are *replaceable adapters* around that spine — swap the model, the knowledge and its provenance stay.\n\n```mermaid\n%%{init: {'theme':'base','themeVariables':{'background':'#FAF3E7','primaryColor':'#FAF3E7','primaryTextColor':'#071812','primaryBorderColor':'#1C4434','lineColor':'#3C5B4E','secondaryColor':'#EDE3D2','tertiaryColor':'#F3EAD9','textColor':'#1C4434','fontFamily':'ui-sans-serif, system-ui, sans-serif'}}}%%\nflowchart LR\n subgraph usual[\"The usual pipeline — lossy, opaque\"]\n direction LR\n R1[Repository] --> E1[\"Embeddings
chunks, no provenance,
drifts from HEAD\"] --> A1[AI Agent]\n end\n subgraph kbflow[\"knowbase — grounded, versioned\"]\n direction LR\n R2[Repository] --> X2[\"Knowledge Extraction
deterministic + LLM adapters\"] --> K2[\"Knowledge Layer
provenance · method ·
confidence · freshness\"] --> A2[AI Agent]\n end\n```\n\n## Why\n\n- **Code is implementation, not knowledge.** \"What are the API contracts?\", \"which module owns billing?\", \"what invalidates this cache?\" are not answered by reading one file — the answer is spread across the repo and lives in nobody's head.\n- **Docs rot.** Hand-written architecture docs and diagrams drift from `HEAD` the moment they are merged. There is no mechanical link back to the code, so nothing tells you when they go stale.\n- **Agents get fragments.** RAG-over-code retrieves nearest-neighbor chunks with no provenance and no notion of whether they still reflect the current commit. The model fills the gaps by guessing.\n\nknowbase answers with **units of knowledge that are versioned against git and traceable to source** — or it does not answer at all.\n\n## How it works — the provenance spine\n\nThe core invariant: **nothing is stored unless it is bound to ≥ 1 exact code span** (`file:line@sha`). That single rule buys three properties at once:\n\n- **Anti-hallucination.** An ungrounded artifact is *not stored* — enforced both in-app (a `GroundingError` before any write) and in the database (a deferred `artifact_grounded_check` constraint trigger that fails the transaction at `COMMIT`). An extractor that cannot point at code cannot persist a claim.\n- **Incremental update.** A `git diff` maps changed code to changed spans, which invalidates *exactly* the derived artifacts whose grounding moved — no over-invalidation, no stale survivors.\n- **Consumer trust.** Every served unit carries its provenance, the method that produced it (deterministic vs. model), confidence, and freshness relative to the commit.\n\nIdentity is **content-addressed and location-free by construction**. A span's `span_id` is a `sha256` over `(normalization_version, lang, span_kind, fq_symbol_path, structural_fingerprint)` — no file path, no byte offsets. The structural fingerprint is a normalized S-expression of the tree-sitter parse (named nodes only; comments *and* docstrings dropped; identifiers and literals kept). So reformatting, moving a file, or editing a comment does **not** change identity; a real rename or a structural edit does. Location is recorded per-SHA in `span_occurrence`, separate from identity.\n\nArtifacts are content-addressed the same way — over their byte-sorted, de-duplicated grounding spans plus the `logical_key` and `extractor_id`/`extractor_version` (and `prompt_version`/`model_id` for model-backed extractors). The `logical_key` is part of the identity so that two different knowledge units sharing their entire evidence set (mutually referencing entities, mutually recursive callers) can never collapse into one digest. Re-indexing the same commit reproduces the identical set of artifact ids.\n\nThe spine is a handful of content-addressed tables: each **artifact** carries ≥ 1 `derived_from` edge to a **code_span**, spans are located **per-SHA** in `span_occurrence`, and a per-SHA **snapshot** ties the grounded artifacts to a commit.\n\n```mermaid\n%%{init: {'theme':'base','themeVariables':{'background':'#FAF3E7','primaryColor':'#FAF3E7','primaryTextColor':'#071812','primaryBorderColor':'#1C4434','lineColor':'#3C5B4E','secondaryColor':'#EDE3D2','tertiaryColor':'#F3EAD9','textColor':'#1C4434','fontFamily':'ui-sans-serif, system-ui, sans-serif'}}}%%\nflowchart TD\n AR[\"artifact
knowledge unit (+ embedding)\"]\n CS[\"code_span
content-addressed id · location-free\"]\n SO[\"span_occurrence
file:line @ sha\"]\n SE[\"snapshot_entry
per-SHA manifest\"]\n CM[\"commit_ref / branch_ref\"]\n AR -->|\"derived_from ≥1 (else rejected:
GroundingError + DB trigger)\"| CS\n CS -->|\"located per-SHA\"| SO\n AR -->|\"appears in\"| SE\n SE -->|\"scoped to\"| CM\n```\n\nIndexing one commit walks that spine end to end — `INGEST → STRUCTURE → INVALIDATE → EXTRACT → SNAPSHOT → SERVE`, with `kb embed` as a separate pass that adds pgvector semantic search on top:\n\n```mermaid\n%%{init: {'theme':'base','themeVariables':{'background':'#FAF3E7','primaryColor':'#FAF3E7','primaryTextColor':'#071812','primaryBorderColor':'#1C4434','lineColor':'#3C5B4E','secondaryColor':'#EDE3D2','tertiaryColor':'#F3EAD9','textColor':'#1C4434','fontFamily':'ui-sans-serif, system-ui, sans-serif'}}}%%\nflowchart LR\n G[\"git blobs @ SHA
no checkout\"] --> S[\"tree-sitter spans
content-addressed identity\"]\n S --> I[\"invalidate
diff-based\"]\n I --> X[\"extractors
deterministic · each grounded ≥1 span\"]\n X --> N[\"per-SHA snapshot
manifest of grounded artifacts\"]\n N --> EV[\"eval gate\"]\n EV --> SV[\"serve
humans + AI agents (MCP)\"]\n N -. \"kb embed (separate pass)\" .-> EM[\"pgvector embeddings
semantic search\"]\n EM -.-> SV\n```\n\n## Status\n\n**v0.8 — spine (artifact-identity rule v2), six deterministic extractors (cross-file grounding, incl. the library public-API surface, event handlers, and call-graph edges) plus the grounded process-path builder over them, LLM-grounded descriptions up to whole-repo architecture overviews and business-process-path labels, ADR-candidate mining from git history (`kb mine`), live incremental re-index (`kb watch`), MCP serving, the fifteen knowledge gates, and published Docker images.** Everything here grounds what it claims, and nothing it cannot:\n\n- **Provenance spine** — content-addressed `span_id` (LOCKED); tree-sitter spans with a normalized S-expression fingerprint and per-SHA location; a single-Postgres, Alembic-managed store with content-addressed idempotent writes; the ≥ 1 `derived_from` anti-hallucination invariant enforced in-app *and* by a deferred DB trigger; pygit2 git ingest (no checkout) with a diff-based invalidation seed.\n- **Deterministic extractors** — the **import / dependency graph** (grimp resolves the edge, tree-sitter grounds it on the exact import statement, with an honest `approximate` fallback for re-exports / relative / unmappable imports — never a silent loss); the **FastAPI API-contract** extractor, which grounds a single route **across files** (handler in `routes.py` + `response_model` class in `schemas.py`); the **domain-entity** extractor (pydantic / dataclass / SQLAlchemy classes and their fields, grounded on the class definition **and cross-file on the entities they reference** — purely static, with documented detection limits); the **library public-API-surface** extractor (what a package exposes from its `__init__.py` — `__all__`-authoritative, with `__init__` re-exports resolved **cross-file** to the defining function/class — validated against an independent **griffe** static oracle); the **event-handler** extractor (pydantic `@field_validator`/`@model_validator`, FastAPI `@app.on_event`, SQLAlchemy `@event.listens_for` **and the module-level call-form `event.listen(Target, \"e\", fn)`** — one artifact per handler, grounded on the handler span, **cross-file** on the class it listens to, and on the registering file for call-form wiring; the bare `from sqlalchemy.event import listen` form is resolved via the import table; function-body/lambda `listen` and dynamic names are documented gaps); and the **call-graph edge** extractor (deterministic caller→callee edges — same-module, imported **cross-file**, and `self.` method calls — one `call_edge` per resolved pair with call-site lines aggregated; `obj.method(...)`, `getattr`, inherited self-calls and decorator/default-arg expressions are documented gaps; the deterministic foundation under the business-process extractor); and the **process-path builder** (second-order: BFS over the call edges from extracted entrypoints — route/event handlers — to sink-registry matches, one `process_path` grounded on EVERY span of the chain, multi-file provenance; built-in sink registry + per-repo `.kb/sinks.yaml` override).\n- **`kb introspect`** — a sandboxed, network-blocked `app.openapi()` oracle, eval-only and never on the index path, that the API gate scores the static contract against.\n- **Read-only MCP server** — `find_provenance`, `get_knowledge`, and `search_knowledge`, each returning provenance-carrying units (method + confidence + freshness).\n- **pgvector embeddings + semantic search** — a replaceable embedding provider (sentence-transformers by default, OpenAI optional) populated by a separate `kb embed` pass; torch stays out of the index path.\n- **A frozen RAG-over-source baseline** and the **Tier-3 knowledge-vs-RAG recall gate** — the honest A/B that backs the \"knowledge > RAG\" thesis.\n- **LLM-grounded descriptions** — an optional, key-gated `kb describe` pass has an LLM write NL summaries for routes, entities, **event handlers**, modules (per file), **packages** (a per-package architecture overview that synthesizes the import graph + public surface + member-module summaries, grounded on the package's own and its direct submodules' spans), **business-process paths** (each materialized `process_path` chain gets an LLM-written label grounded on every span along the path, with a confidence that honestly stays < 1.0), and **the repo as a whole** (one whole-repo overview per snapshot, grounded on the bounded top-level surface and synthesizing the package overviews beneath it); every claim is validated against the target's own spans by a deterministic sub-property gate, so ungrounded claims are *dropped* (the anti-hallucination invariant, with a model in the loop). Stored as `extraction_method = \"llm_grounded\"`, grounded on code spans.\n- **Incremental re-index** — `kb index --parent ` reuses unchanged files' spans from the parent snapshot and parses only the diff; extraction stays full, so the result is identical to a full re-index (a HARD gate proves it). Auto-detects the parent; falls back to full when none is indexed.\n- **ADR-candidate mining** — an optional, key-gated `kb mine` pass walks local first-parent git history and has an LLM extract the *decision* a commit records (the \"why\" that code alone cannot show). The provenance bridge: each `decision` artifact is grounded on the spans its source commit **changed** — prose-born knowledge still points at exact code spans; the commit message is stored verbatim as a fact (immutable, pinned by the sha), and every claim passes the same deterministic span-validation floor (fabricated claims dropped; a commit with no surviving claim stores nothing). With **`--prs`** (strictly opt-in — the only networked step in the toolchain), a squash-merged commit's GitHub PR title/body enrich the prompt and the payload; PR text stays context, never grounding, and a digest of it is folded into the artifact's identity, so a PR edited after merge yields a *new* artifact instead of a silent payload swap.\n- **Fifteen HARD CI eval gates** (see [Development](#development)).\n\n- **A nightly LLM-judged A/B** (optional, key-gated, **non-gating**) — an answerer LLM answers each question from knowbase's grounded context vs a RAG-over-source context, and a judge LLM scores **answer accuracy** (against hand-written gold) + **hallucination**. Tracked metrics on top of recall; it never blocks CI.\n\n**Not done yet** (and deliberately not faked): languages beyond Python. See the [Roadmap](#roadmap).\n\n## Quickstart\n\n### Prerequisites\n\n- **Python 3.12+**\n- **uv** ([install](https://docs.astral.sh/uv/getting-started/installation/))\n- **PostgreSQL 17** — required to *run the daemon*. For the **test suite** you do not need a running server: it spins an **ephemeral local Postgres cluster** via `initdb`/`pg_ctl` (no Docker). You just need the Postgres *binaries* on the machine (e.g. from Postgres.app or a system package). Point at them with `KB_PG_BINDIR` if they are not on `PATH`, or skip the ephemeral cluster entirely by setting `KB_TEST_DB_URL` to an existing database.\n\n### Install\n\n```bash\nuv sync --extra dev # create the venv + install everything\nuv sync --extra dev --extra embed # add the embedding stack (CPU torch) for `kb embed` + semantic search\nuv sync --extra dev --extra llm # add the LLM SDKs for `kb describe` / `kb mine`\n```\n\nThe base `--extra dev` install stays torch-free; the `embed` extra pulls sentence-transformers (CPU-only torch via a pinned index) and is only needed for `kb embed` and `search_knowledge`.\n\n### Run the gates\n\n```bash\nuv run pytest src/kb/eval -q # the fifteen HARD gates (spins an ephemeral local Postgres)\n```\n\n### Index a commit\n\n```bash\nuv run kb --help\nuv run kb index --sha --db-url \n\n# incremental: reuse unchanged files' spans from an already-indexed parent (parse only the diff)\nuv run kb index --sha --parent --db-url \n```\n\n`--sha` accepts any commit-ish (sha, branch, tag, or `HEAD`, the default). The database URL can also come from the `KB_DB_URL` environment variable instead of `--db-url`. A run prints what it produced:\n\n```text\nindexed 4f1c2a9b8d3e (full): 12 files (12 parsed, 0 reused), 318 spans, 27 artifacts, 1 gaps\n gaps (unparseable, recorded): src/legacy/broken.py\n```\n\n**Incremental re-index** (`--incremental`, or `--parent ` which implies it) reuses the spans of files unchanged since the parent snapshot and parses only changed/new files; the extractors still run fully, so the snapshot is identical to a full re-index (a HARD gate proves it). The parent is auto-detected from the commit's parents (the first already-indexed one); a missing/unindexed parent or a first-party-root change falls back to a full index. This makes `kb index` cheap to wire to a git `post-receive` hook or a CI step: `kb index --sha --parent `.\n\n### Watch a repo\n\n```bash\nuv run kb watch --branch main --interval 30 --db-url \nuv run kb watch --once # one tick and exit (cron/CI-friendly)\n```\n\n`kb watch` polls a **local** branch ref (no network, no credentials) and indexes every new first-parent commit incrementally, recording its resume point in the `branch_ref` table — the cursor advances per indexed commit, so a restart resumes where it left off. When the branch jumped further than `--max-catchup` (default 50) or was force-pushed, it indexes just the new head against the recorded cursor. Pair it with whatever moves the ref: a **bare repo receiving pushes** (run the watcher next to it), a **cron `git pull`** in a clone, or a **CI step** using `--once`. One database tracks one repository.\n\nUnder the hood it runs the spine for that one commit — `INGEST → STRUCTURE → EXTRACT → SNAPSHOT`. For example, an import like `from shop.billing import charge` on line 1 of `shop/orders.py` becomes an `import_edge` artifact (`import:shop.orders->shop.billing`) grounded on the exact `import` span at that `file:line@sha`, with `span_mapping: \"exact\"`. **\"Gaps\"** are files that hit a syntax error: they are *recorded*, never silently dropped, so blind spots are visible rather than invisible.\n\n### Add semantic search\n\n```bash\nuv run kb embed --db-url # separate pass: populate artifact embeddings\n```\n\n`kb embed` runs a replaceable embedding provider (sentence-transformers `all-MiniLM-L6-v2` by default, OpenAI optional via `KB_EMBED_PROVIDER=openai`) over the latest snapshot's artifacts and writes them into `artifact.embedding` (pgvector). It is idempotent and torch only loads when this command runs — never on the index path.\n\n### Generate semantic descriptions (LLM-grounded)\n\n```bash\nuv run kb describe --db-url # separate, key-gated pass (ANTHROPIC_API_KEY / OPENAI_API_KEY)\n```\n\n`kb describe` has an LLM (via `kb.llm`, `KB_LLM_PROVIDER` in {`anthropic`,`openai`}) write a short NL summary + structured claims for each route, entity, **event handler**, module (per file), **package** (a per-package architecture overview, grounded on the package's own and its direct submodules' spans, synthesizing the import graph + public surface + member-module summaries as context), **business-process path** (a grounded NL label for the deterministic `process_path` chain, grounded on every span along the path), and **the repo as a whole** (one overview per snapshot, grounded on the bounded top-level surface, synthesizing the package overviews) in the latest snapshot. **Every claim is validated against that target's own grounding spans** — claims citing a symbol not in the code are dropped, and a `description` artifact is stored only if something survives, grounded on those code spans (`extraction_method = \"llm_grounded\"`). It needs an API key (and the `llm` extra), never runs on `kb index`, and the deterministic grounding gate is exercised in CI without a key (stub LLM). `KB_LLM_MODEL` overrides the provider's default model, and any OpenAI-compatible endpoint works via the SDK's `OPENAI_BASE_URL` — e.g. OpenRouter: `KB_LLM_PROVIDER=openai OPENAI_BASE_URL=https://openrouter.ai/api/v1 KB_LLM_MODEL=anthropic/claude-sonnet-4.6 OPENAI_API_KEY=`.\n\n### Mine decision candidates from git history (LLM-grounded)\n\n```bash\nuv run kb mine --db-url # separate, key-gated pass (ANTHROPIC_API_KEY / OPENAI_API_KEY)\n```\n\n`kb mine` walks the local first-parent history from the latest indexed commit (`--sha` to start elsewhere, `--max-commits` to bound the LLM spend) and has an LLM extract the decision each commit records from its message plus its changed code. Each stored `decision` artifact is **grounded on the spans the commit changed** — the same ≥ 1-span invariant as everything else — with the commit message kept verbatim in the payload as a fact. Claims are span-validated (fabricated claims dropped; nothing survives → nothing stored); merge commits are skipped (squash-merge flow is fully covered; grounding a classic merge on its first-parent diff is an open question), docs-only commits never pay an LLM call, and re-running skips already-mined commits (`--force` to re-mine). Requires the commits to be indexed first (`kb index` / `kb watch`).\n\nAdd `--prs` to enrich decisions with GitHub PR titles/bodies (`GITHUB_TOKEN=... kb mine --prs`) — strictly opt-in, the only networked step in the toolchain (anonymous works for public repos at 60 req/h; `--repo-slug owner/name` overrides the origin-derived slug). A digest of the fetched PR text is identity-bearing, and a failed fetch degrades to plain mining with a per-run counter.\n\n### Serve to an AI agent (MCP)\n\n```bash\nuv run kb serve --db-url # read-only MCP server over stdio\n```\n\nThe server exposes three read-only tools, each returning provenance-carrying knowledge units (extraction method + confidence + freshness):\n\n| Tool | Purpose |\n| --- | --- |\n| `find_provenance(file, line, sha?)` | What grounded knowledge sits at `file:line@sha` — the spans there and the artifacts derived from them. |\n| `get_knowledge(target, sha?, token_budget?)` | Resolve a logical key / file / module to its knowledge units, ranked and trimmed to a token budget (omissions reported, never silently truncated). |\n| `search_knowledge(query, sha?, k?, token_budget?)` | Cosine-ranked semantic search over the embedded artifacts (requires `kb embed`). |\n\n### Cross-check the API contract (eval-only)\n\n```bash\nuv run kb introspect app.main:app --repo # sandboxed app.openapi() oracle\n```\n\n`kb introspect` runs a FastAPI app in a network-blocked sandbox and emits its `openapi()` as JSON — the ground truth the Tier-1 API gate scores the **static** contract extractor against. It executes user code, so it is eval-only and never runs during indexing.\n\n## Run with Docker\n\nPrebuilt images are published to **GHCR**: `ghcr.io/v0ropaev/knowbase` (`:edge` from `master`, `:X.Y.Z`/`:latest` on releases; multi-arch amd64+arm64). The default image is **slim** (no torch — `index` / `migrate` / `serve` / `introspect`); the **`-embed`** tag (e.g. `:edge-embed`) adds CPU-torch for `kb embed` and semantic search. `kb describe` / `kb mine` need the LLM SDKs, which no published tag carries — build your own with `--build-arg EXTRAS=\"--extra llm\"` (the CLI tells you exactly this if the SDK is missing).\n\n```bash\ndocker pull ghcr.io/v0ropaev/knowbase:latest\ndocker run --rm ghcr.io/v0ropaev/knowbase:latest --help\n```\n\nAs an **MCP server** (stdio), pointed at your Postgres — this is the form an MCP client launches:\n\n```bash\ndocker run -i --rm ghcr.io/v0ropaev/knowbase:latest serve --db-url \n```\n\n### Local dev/eval with `docker compose`\n\nThe bundled compose brings up a `pgvector` Postgres plus the `kb` CLI built from this checkout:\n\n```bash\ndocker compose up -d db # Postgres (pgvector)\ndocker compose run --rm kb migrate # apply the schema\ndocker compose run --rm kb index /workspace --sha HEAD # index the mounted repo\ndocker compose run --rm -i kb serve # MCP over stdio\n```\n\nThe compose Postgres also backs the test suite from the host: `KB_TEST_DB_URL=postgresql+psycopg://postgres:postgres@localhost:5432/postgres uv run pytest src/kb/eval -q`. For embeddings, build the image with the embed extra: `docker compose build --build-arg EXTRAS=\"--extra embed\" kb`.\n\n## Architecture\n\nA Python package `kb` (uv, src-layout). Modules and their responsibilities:\n\n| Module | Responsibility |\n| --- | --- |\n| `kb.ids` | Content-addressed identity hashing (**LOCKED**). `span_id` excludes file path and byte offsets; `artifact_id` refuses to be computed without ≥ 1 grounding span. |\n| `kb.structural` | tree-sitter span extraction; the structural fingerprint is a normalized S-expression (named nodes only; comments and docstrings dropped; identifiers/literals kept). Location is recorded per-SHA. |\n| `kb.store` | A single PostgreSQL via Alembic; content-addressed idempotent writes; the ≥ 1 `derived_from` invariant enforced in-app and by a deferred constraint trigger. |\n| `kb.git` | pygit2 ingest — reads blobs at a SHA (no checkout) — plus the diff-based invalidation seed. |\n| `kb.extract.deterministic.imports` | Deterministic import / dependency edges: tree-sitter spans grounded by line, grimp edge resolution. |\n| `kb.extract.deterministic.fastapi_contract` | Static FastAPI API-contract extractor; grounds a route across files (handler + `response_model` class), never imports user code. |\n| `kb.extract.deterministic.entities` | Static domain-entity extractor — pydantic / dataclass / SQLAlchemy classes + their fields, grounded on the class definition **and, across files, on the entities they reference** (field types + `relationship()`); detection signals and limits recorded in the payload. |\n| `kb.extract.deterministic.library_surface` | Static library public-API-surface extractor — one `public_symbol` per name a package exposes from its `__init__.py` (`__all__`-authoritative), `__init__` re-exports resolved **cross-file** to the defining function/class; validated by an independent griffe static oracle. Never imports user code. |\n| `kb.extract.deterministic.calls` | Static call-graph edge extractor — one `call_edge` per resolved caller→callee pair (same-module / imported **cross-file** / `self.` method), call-site lines aggregated; only resolved first-party edges are emitted; dynamic/attribute/inherited calls are documented gaps. Never imports user code. |\n| `kb.extract.deterministic.paths` | Second-order process-path builder — `process_path` per (entrypoint, sink, terminal): shortest chain of resolved call edges from an extracted route/event handler to a sink-registry match, grounded on every span along the path (multi-file); built-in registry + `.kb/sinks.yaml` override (registry digest is identity-bearing); caps flagged, never silent. |\n| `kb.extract.deterministic.events` | Static event-handler extractor — one `event_handler` per handler carrying pydantic validator / FastAPI `on_event` / SQLAlchemy `listens_for` decorator registrations **and module-level call-form `event.listen(...)` registrations** (all aggregated in `payload.registrations`; the handler may live in another module than the `listen()`), grounded on the handler span, **across files on the class it listens to**, and on the registering file's module span (role `registration_site`); the bare `from sqlalchemy.event import listen` form is resolved via the import table; function-body/lambda `listen` and dynamic names are documented gaps. |\n| `kb.introspect` | Sandboxed, network-blocked `app.openapi()` oracle — eval-only ground truth for the API gate, never on the index path. |\n| `kb.mcp` | Read-only MCP server and its provenance-carrying records: `find_provenance`, `get_knowledge`, `search_knowledge`. |\n| `kb.embed` | Replaceable embedding adapters (sentence-transformers default, OpenAI optional) + snapshot population. Torch isolated behind the `embed` extra and a lazy import. |\n| `kb.rag` | The frozen pgvector RAG-over-source baseline — the \"other arm\" of the knowledge-vs-RAG A/B (no provenance, no grounding). |\n| `kb.extract.semantic` | LLM-grounded extraction (`kb describe`): NL descriptions of routes/entities/event handlers/modules, **per-package architecture overviews** (rich synthesis from the import graph + public surface + member summaries, grounded on code spans), **business-process-path labels** (one per `process_path`, grounded on every span along the path), **and a whole-repo overview** (one per snapshot, grounded on the bounded top-level surface, synthesizing package summaries + cross-package imports + artifact counts) with a deterministic sub-property gate (`grounding.validate_claims`) that drops any claim not backed by the target's spans. Plus **`kb mine`** — ADR-candidate mining over local first-parent git history: one `decision` artifact per mined commit, grounded on the commit's **changed spans**, message verbatim in the payload, claims under the same floor. Both are separate key-gated passes; never on `index`. |\n| `kb.daemon.cli` | The `kb` CLI: `index` (full or `--incremental`/`--parent`), `watch` (poll a local branch ref, per-commit incremental catch-up, `branch_ref` cursor), `migrate`, `embed`, `describe`, `serve` (MCP), and `introspect` — all functional. |\n| `kb.eval` | Fifteen HARD CI gates (identity reproducibility, adversarial grounding, Tier-1 import oracle, Tier-1 API oracle, Tier-1 entities oracle, Tier-1 library-surface oracle, Tier-1 events oracle, Tier-1 call-graph oracle, Tier-1 process-paths oracle, Tier-3 knowledge-vs-RAG recall, Tier-4 one-hop invalidation, invariants, semantic grounding floor, ADR-mining floor, incremental re-index equivalence) plus the supporting MCP / embed / store suite. |\n\nCore tables: `commit_ref`, `branch_ref`, `code_span`, `span_occurrence`, `artifact` (now with `embedding vector(384)` + `embedding_model_id`), `artifact_derived_from`, `snapshot_entry`, and `rag_chunk` (the baseline arm).\n\n## Development\n\n```bash\nuv sync --extra dev # venv + install\nuv run ruff check src/kb # lint\nuv run mypy # strict type-check\nuv run pytest src/kb/eval -q # the fifteen HARD eval gates\n```\n\nCI (GitHub Actions, workflow **\"CI\"**, `.github/workflows/ci.yml`) runs ruff, `mypy --strict`, and the eval gates against a `pgvector/pgvector:pg17` service (with the embedding model cached). The **fifteen HARD gates** that block a merge:\n\n1. **Identity reproducibility** — formatting / comment / docstring / location changes must NOT change `span_id`; a rename MUST. Pure identity core, no database.\n2. **Adversarial grounding** — an ungrounded artifact is rejected by *both* layers (the app's `GroundingError` and the DB's deferred `artifact_grounded_check` trigger); a genuinely grounded artifact commits cleanly.\n3. **Tier-1 import oracle** — extracted import edges match a hand-labeled oracle, grounded on the actual import statement span; a dynamic import is asserted as a *known* gap, not a silent loss.\n4. **Tier-1 API oracle** — the statically-extracted FastAPI contract equals the app's own `openapi()` (from the sandboxed introspect oracle), and the route's cross-file grounding (handler + `response_model`) is asserted.\n5. **Tier-1 entities oracle** — extracted pydantic / dataclass / SQLAlchemy entities + their fields match a hand-labeled oracle, each grounded on its class span; a bare declarative `Base` is correctly *not* an entity and a `create_model(...)` model is asserted as a *known* gap.\n6. **Tier-1 library-surface oracle** — the statically-extracted public API surface of a package (`__all__`-authoritative, `__init__` re-exports resolved **cross-file**) equals an independent **griffe** static oracle; underscore-private names are excluded, and a third-party / dynamic-`__all__` re-export is grounded-but-flagged, asserted as a *known* gap.\n7. **Tier-1 events oracle** — extracted event handlers (pydantic validators, FastAPI `on_event`, SQLAlchemy `listens_for` and the module-level call-form `event.listen`) match a hand-labeled oracle, each grounded on its handler span and **cross-file** on the class it listens to; a call-form registration wired in a third module grounds ONE artifact across **three files** (handler + target class + registration site); `listen(...)` inside a function body, a lambda listener, and a dynamic `@app.on_event(EVENT)` name are asserted as *known* gaps.\n8. **Tier-1 call-graph oracle** — extracted call edges match a hand-labeled oracle (same-module, cross-file imported — incl. aliases and module-attribute forms — `self.` method calls, and class instantiation), each grounded on the caller **and** callee def spans (cross-file); a mutually recursive pair yields two distinct artifacts (identity-v2 regression); `obj.method(...)`, `getattr`, inherited self-calls and decorator-expression calls are asserted as *known* gaps.\n9. **Tier-3 knowledge-vs-RAG recall** — knowbase cross-file recall@k == 1.0 for every cross-file question (API contracts **and** domain entities: in each case one artifact already spans both files, so the floor is *structural*, independent of embedding quality); the RAG arm is reported but **never asserted**, so a model bump can't redden CI.\n10. **Tier-4 one-hop invalidation** — a content diff invalidates *exactly* the artifacts whose grounding span changed (set-equality: no over-invalidation, no stale survivors); a version bump invalidates everything.\n11. **Invariants** — zero orphans (every snapshot artifact is grounded), and re-indexing the same SHA yields the identical set of artifact ids.\n12. **Semantic grounding floor** — the LLM-grounded describer's claims are validated against the artifact's own spans by a deterministic sub-property gate; an adversarial fabricated claim is *dropped*, never stored, on every describe path (artifacts incl. event handlers, modules, packages, process paths, and the whole-repo overview — whose grounding is provably bounded: a grandchild module never grounds it). Run on a stub LLM, so it gates without an API key.\n13. **Incremental re-index equivalence** — an incremental re-index (reuse unchanged files' spans from the parent, parse only the diff) yields the *identical* `{logical_key: artifact_id}` snapshot as a full re-index of the same tree, and the parse is provably skipped for unchanged files (counter assertions); a missing/unindexed parent falls back to full.\n14. **Tier-1 process-paths oracle** — materialized business-process paths match a hand-labeled oracle (2-hop cross-file chain, 0-hop direct-sink handler, event-handler entrypoint); the flagship artifact is grounded across **three files**; a reachable call cycle cannot hang the BFS; the fixture's own `.kb/sinks.yaml` proves the override chain; a no-sink route yields no artifact; depth caps are honored.\n15. **ADR-mining floor** — mined `decision` artifacts are grounded on the spans their source commit changed (role `changed`, provenance limited to the touched files); an adversarial fabricated claim is *dropped*; a commit whose changed code backs no claim stores nothing; a docs-only commit never calls the LLM; merges are skipped; re-mining is idempotent and never re-bills stored decisions. PR enrichment is covered too: the PR-text digest is identity-bearing (an edited PR yields a new id, never a silent swap), a failed fetch degrades to the byte-identical plain artifact, and commits without a PR reference stay byte-compatible (run on stub LLM/PR providers, so it gates without keys or network).\n\nThe identity rules in `kb.ids` (and `kb.structural`) are **LOCKED**: changing one is a breaking change, gated behind a `NORMALIZATION_VERSION` / `extractor_version` bump so existing digests are invalidated rather than silently colliding.\n\n## Roadmap\n\nThe honest north star is to show that a grounded **knowledge layer beats RAG-over-code** on real questions. The reason it can is structural: to answer a cross-file contract question, RAG must independently retrieve *two* source chunks across files, while one grounded knowbase artifact already spans both — which is exactly what the Tier-3 gate measures.\n\n```mermaid\n%%{init: {'theme':'base','themeVariables':{'background':'#FAF3E7','primaryColor':'#FAF3E7','primaryTextColor':'#071812','primaryBorderColor':'#1C4434','lineColor':'#3C5B4E','secondaryColor':'#EDE3D2','tertiaryColor':'#F3EAD9','textColor':'#1C4434','fontFamily':'ui-sans-serif, system-ui, sans-serif'}}}%%\nflowchart LR\n Q[\"Q: shape of GET /api/orders response?\"]\n Q --> A[\"api:GET /api/orders
(one grounded artifact)\"]\n Q --> RAG\n A --> H[\"routes.py — handler span ✓\"]\n A --> M[\"schemas.py — response_model span ✓\"]\n subgraph RAG[\"RAG-over-source — independent top-k chunks\"]\n direction TB\n C1[\"chunk · routes.py ✓\"]\n C2[\"chunk · schemas.py — only if retrieved\"]\n C3[\"chunk · other.py ✗\"]\n end\n```\n\nNext milestones:\n\n- [x] **Nightly LLM-judged A/B** (key-gated, non-gating) — grounded-answer accuracy + hallucination rate on top of recall. *(shipped)*\n- [x] **LLM-grounded semantic layer** — model-backed artifacts that still carry ≥ 1 span (`extraction_method = \"llm_grounded\"`): `kb describe` writes span-validated descriptions for routes, entities, event handlers, modules, per-package architecture overviews, and a whole-repo overview. *(shipped)*\n- [x] **Incremental re-index on git push** — `kb index --incremental`/`--parent` reuses unchanged files' spans from the parent snapshot (extraction stays full; equivalence is gated), and `kb watch` polls a local branch ref, indexing each new first-parent commit incrementally with a `branch_ref` cursor (`--max-catchup` guard, `--once` for cron/CI). *(shipped)*\n- [x] **ADR mining — slice 1 (local commit history)** — `kb mine` walks first-parent history and stores span-grounded `decision` candidates: each grounded on the spans its commit changed, message verbatim as a fact, claims under the deterministic span-validation floor. *(shipped)*\n- [x] **PR-description mining** — `kb mine --prs` (strictly opt-in, the only networked step) enriches decisions with GitHub PR titles/bodies for squash-merged commits; the PR-text digest is identity-bearing, so post-merge PR edits can never silently swap a payload. *(shipped)*\n- [x] **Grounded business-process extraction** — `process_path` artifacts (entrypoint → call chain → sink-registry match, multi-file grounding) plus LLM labeling via `kb describe` (a span-validated NL label per path, grounded on the path's own spans, confidence honestly < 1.0). *(shipped)*\n- [ ] **More languages** beyond Python.\n\n## Contributing\n\nContributions are welcome. See [CONTRIBUTING.md](./CONTRIBUTING.md) for setup, the eval-gate discipline, and pull-request expectations. A useful rule of thumb: if an extractor cannot ground a claim on a code span, it does not get to make the claim.\n\n## Security\n\nPlease report vulnerabilities responsibly — see [SECURITY.md](./SECURITY.md). Do not open public issues for security reports.\n\n## License\n\n[AGPL-3.0-or-later](./LICENSE).\n"}