prbe-knowledge

Probe Knowledge Engine

Probe is a self-hosted knowledge engine. It ingests your team's activity from GitHub, Slack, Linear, Notion, Sentry, and custom sources; normalizes, chunks, and embeds it into Postgres; and serves hybrid retrieval (vector + keyword + graph) plus LLM-synthesized, cited answers and auto-generated knowledge pages. This repository is the open-source community edition: a single-tenant engine you run yourself, with no control plane and no calls out to Probe's hosted service.

30-second quickstart

cp .env.example .env
# In .env, fill three things:
#   GOOGLE_API_KEY        — embeddings (Gemini)
#   ANTHROPIC_API_KEY     — or OPENAI_API_KEY, for answer synthesis
#   KNOWLEDGE_API_TOKEN   — your query bearer token (openssl rand -hex 32)

make up        # build images + start the full stack (or: docker compose up)
make health    # confirm ingestion + retrieval are live

Then ask a question:

curl -X POST http://localhost:8081/query \
  -H "Authorization: Bearer $KNOWLEDGE_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query": "what changed last week?"}'

make up brings up Postgres (pgvector), MinIO (object store), a one-shot DB migration, then the ingestion, retrieval, and worker services. In single-tenant community mode everything is scoped to DEFAULT_CUSTOMER_ID (defaults to default) and gated by KNOWLEDGE_API_TOKEN. See docs/self-hosting.md for the full setup.

Apple Silicon note: the Gemini embedding SDK's native grpcio wheel can crash the ingestion/worker containers with SIGILL (exit 132) on arm64. Run on linux/amd64 — export DOCKER_DEFAULT_PLATFORM=linux/amd64 before make up, or deploy on an amd64 host. amd64 is unaffected. See docs/self-hosting.md.

How it works

                    ┌──────────────────────────────────────────────┐
   data sources     │           Probe knowledge engine             │
 ─────────────────  │                                              │
  GitHub  Slack     │   ┌─────────────┐         ┌──────────────┐   │
  Linear  Notion ──▶│   │  ingestion  │  enqueue│    worker     │   │
  Sentry  custom    │   │  :8080      │────────▶│ normalize ·   │   │
                    │   │  webhooks + │         │ chunk · embed │   │
                    │   │  custom API │         └──────┬───────┘   │
                    │   └──────┬──────┘                │           │
                    │          │ raw payload           │ docs +    │
                    │          ▼                        ▼ chunks +  │
                    │   ┌──────────────┐        ┌──────────────┐   │
                    │   │ object store │        │  Postgres 16  │   │
                    │   │ (S3/R2/MinIO)│        │  + pgvector   │   │
                    │   │ raw archives │        │  docs·chunks· │   │
                    │   └──────────────┘        │  graph·queue  │   │
                    │                           └──────┬───────┘   │
                    │   ┌─────────────┐                │           │
   agents / apps ──▶│   │  retrieval  │◀───────────────┘           │
   ◀── answers      │   │  :8081      │  vector + BM25 + graph      │
                    │   │  /query     │  → RRF fusion → synthesis   │
                    │   │  /retrieve  │                             │
                    │   └─────────────┘                             │
                    └──────────────────────────────────────────────┘

• Ingestion (:8080) takes signed provider webhooks at POST /webhooks/{source} (verified in-process by each connector) plus a custom-ingest API, persists the raw payload to object storage, and enqueues work. GET /health for liveness.
• Worker drains the queue: normalize → chunk → embed → upsert documents, chunks, and the entity graph into Postgres.
• Retrieval (:8081) serves POST /retrieve (raw chunks: vector + BM25 + graph fused via RRF) and POST /query (retrieval + LLM-synthesized cited answer). GET /health for liveness.

Full design: docs/phase0-design.md · storage/data flow: docs/storage-architecture.md · how retrieval works: docs/retrieval-architecture.md.

Retrieval quality

The agent retrieval path (/retrieve) is measured against a held-out conversational-memory benchmark — a blend of LongMemEval, LoCoMo, and two in-house graph suites (gbrain, relational) — scored as macro-averaged recall@10 across 12 axes.

A recent pass with our automated optimization harness (auto-research) lifted macro-avg recall@10 from 0.412 → 0.689 (+0.28 absolute, ~67% relative) by stacking four changes on the gatherer:

1. Wider candidate window — vector + BM25 top-k 15 → 30 (the two channels that carry recall when a query has no graph entity to anchor on).
2. Multi-query fan-out — the extractor proposes up to 3 complementary reformulations (multi-hop decomposition, source-vocabulary rephrasing, temporal/event focus) that run alongside the raw query in one pre-fan-out.
3. RRF recall-floor backfill — every (sub-query × channel) ranked list is reciprocal-rank-fused, and the top fused docs the single-turn gatherer dropped are appended until ≥10 distinct docs reach the consumer.
4. Recall-first curation — the gatherer is prompted to emit every plausibly-relevant turn, especially across multiple sessions and over time.

These are latency-neutral: on this path the gatherer LLM is the reranker (raw channel hits go straight into its prompt), and graded latency is set by the number of sequential LLM turns — so widening the single-turn candidate pool adds no round-trips.

Axis	Baseline	Optimized	Δ
Macro-avg (combined)	0.412	0.689	+0.277
LongMemEval · single-session-preference	0.188	0.875	+0.688
LongMemEval · temporal-reasoning	0.094	0.708	+0.615
LongMemEval · multi-session	0.458	0.833	+0.375
LoCoMo · multi-hop	0.229	0.576	+0.347
LongMemEval · knowledge-update	0.469	0.812	+0.344
LoCoMo · temporal	0.333	0.667	+0.333
LoCoMo · commonsense	0.042	0.292	+0.250
LoCoMo · single-hop	0.333	0.583	+0.250
LongMemEval · single-session-user	0.250	0.438	+0.188
gbrain · brainbench-cat13	0.800	0.830	+0.030
LongMemEval · single-session-assistant	1.000	1.000	+0.000
relational · who-rel	0.747	0.657	−0.090

Gains concentrate on the weak conversational-memory axes. The one regression (relational/who-rel, −0.09) is a deliberate trade — broader recall for slightly lower precision on an already-strong, entity-anchored axis. (Macro-avg nDCG@10 also rose, 0.459 → 0.556.)

Repo layout

prbe-knowledge/
├── engine/             content-agnostic core (never imports kb/)
│   ├── ingest/         queue drain · normalize · chunk · embed, plus the
│   │   │               generic ingest doors (custom-ingest API, manual
│   │   │               uploads) and the connector contract + registry
│   │   └── handlers/   Connector ABC, registry, engine-door connectors
│   ├── retrieval/      /retrieve + /query (:8081): retrievers + RRF fusion
│   ├── mcp/            MCP server: agent tool surface (:8084)
│   ├── community/      Leiden graph clustering
│   └── shared/         config, db/tenancy, storage, models, enums,
│                       source registry
├── kb/                 integrations domain (imports engine/)
│   ├── handlers/       one connector per source (Slack, GitHub, ...)
│   ├── polling/        outbound pollers (self-host mode)
│   ├── ingestion_app.py  webhook + ingest HTTP app (:8080)
│   └── worker.py       composed worker process (drain + backfills + pollers)
├── services/           thin deploy wrappers — compose/Helm entrypoints
│                       (services.ingestion.main:app etc.) stay stable
├── db/
│   └── schema.sql      canonical Postgres schema (source of truth)
├── scripts/
│   └── migrate.py      generic DB bootstrap (schema.sql + alembic)
├── deploy/helm/        community Helm chart (probe-knowledge)
├── docker-compose.yml  turnkey local/self-host stack
├── Makefile            up · down · health · query · seed
└── docs/
    ├── self-hosting.md prerequisites, compose, Helm, Postgres, providers
    ├── connectors.md   per-source setup (GitHub App, webhook secrets)
    ├── phase0-design.md
    ├── storage-architecture.md   how data is stored at rest
    └── retrieval-architecture.md how a query becomes a cited answer

Documentation

• Self-hosting — Docker Compose, Helm, Postgres, model providers, object storage, backups.
• Connectors — per-source setup, including how to register your own GitHub App and where each webhook signing secret comes from.
• Contributing — dev setup, gates, PR norms.
• Design · Storage architecture · Retrieval architecture

MCP server

The agent tool surface (search_knowledge, query_knowledge, get_source) ships in-repo at engine/mcp/ (vendored from the former standalone prbe-knowledge-mcp). It proxies to the retrieval service over HTTP (KNOWLEDGE_QUERY_URL) and has two auth modes via MCP_AUTH_MODE:

• static (community self-host) — one shared bearer (MCP_API_TOKEN) scoped to DEFAULT_CUSTOMER_ID. This is the docker compose default; point your client at http://localhost:8084/mcp with Authorization: Bearer $MCP_API_TOKEN.
• oauth (Probe-hosted) — validates OAuth 2.1 JWTs against the issuer JWKS (MCP_OAUTH_*).

Mode auto-resolves from the env (static when only MCP_API_TOKEN is set, oauth when a JWKS URL is set); set MCP_AUTH_MODE to force one.

The MCP and installer guidance uses a decision gate: agents search only when a concrete question about team history could change the answer or approach, such as prior rationale, incidents, ownership, constraints, or parallel work. Routine implementation, review, status, and shipping steps do not trigger a lookup. Re-running the hosted /install script refreshes this shared guidance for Codex, Claude Code, and Cursor while preserving unrelated instructions.

Contributing & community

• Contributing — dev setup, gates, PR norms.
• Code of Conduct — expectations for participation.
• Security policy — how to report a vulnerability privately.
• Changelog — notable changes per release.

License

AGPL-3.0. Copyright © 2026 prbe-ai.

This repository is the canonical engine and the open-source community edition — a single-tenant engine you run yourself. Probe's hosted multi-tenant control plane (per-tenant routing, gateway trust, usage metering) is intentionally dormant in this build, not removed; the community edition makes no calls out to Probe's hosted service.