{"id":"0357a0d7-e0ce-4cec-b341-2d1418b3cbbe","slug":"agentic-data-kernel","kind":"mcp","name":"agentic-data-kernel","summary":"Agent-first persistence for temporal knowledge, durable workflows, causal lineage, and controlled external effects","owner":"Jason-Doyle","repoUrl":"https://github.com/Jason-Doyle/agentic-data-kernel","license":"Apache-2.0","licenseTier":"permissive","trust":"community","stars":2,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-09-02T06:20:55.000Z","addedAt":"2026-10-21T09:36:55.372Z","logoUrl":"https://avatars.githubusercontent.com/u/46789294?u=416410d825ee3b94e4337b9b321cb8a1b5a9c8a6&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["databases","data-analytics","memory-knowledge"],"tags":["agentic-ai","ai-agents","bitemporal","data-lineage","durable-workflows","pgvector","postgresql","provenance"],"runtime":"node","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-21T09:38:29.168Z","sample":false,"url":"https://conduits.dev/tools/agentic-data-kernel","language":"TypeScript","transports":["stdio"],"install":{"npm":"agentic-data-kernel"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# Agentic Data Kernel\n\n> **Author note, 2 September 2026, [Jason Doyle](https://jasondoyle.ie/)**\n>\n> This project began as a feature forked from a private project and is now\n> maintained independently as open source. The documentation is heavily AI\n> assisted and may contain errors. Verify important behavior against the\n> implementation, tests, and current release before using it in production.\n\n[![CI](https://github.com/Jason-Doyle/agentic-data-kernel/actions/workflows/ci.yml/badge.svg)](https://github.com/Jason-Doyle/agentic-data-kernel/actions/workflows/ci.yml)\n[![CodeQL](https://github.com/Jason-Doyle/agentic-data-kernel/actions/workflows/codeql.yml/badge.svg)](https://github.com/Jason-Doyle/agentic-data-kernel/actions/workflows/codeql.yml)\n[![npm](https://img.shields.io/npm/v/agentic-data-kernel?label=npm)](https://www.npmjs.com/package/agentic-data-kernel)\n[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)\n\nCreated and maintained by [Jason Doyle](https://jasondoyle.ie/).\n\nAn agent-first persistence layer for durable knowledge, workflows, and\ncontrolled external effects.\n\nMost databases are good at storing current application state. Long-running\nagents also need to preserve:\n\n- what was known at a particular time;\n- which evidence supported or contradicted it;\n- why a decision was authorized;\n- which workflow revision produced an external action;\n- whether a timed-out action happened;\n- how later reconciliation and verification changed the outcome.\n\nAgentic Data Kernel provides those semantics over SQLite for local development\nand PostgreSQL for production. It is not a new storage engine, a replacement\nfor SQL, or a claim that PostgreSQL cannot implement the same behavior. It is a\nshared contract for patterns that agentic applications otherwise rebuild as\napplication-specific tables, workflow code, lineage queries, and effect\nrecovery logic.\n\n## Proof before platform\n\nThe flagship scenario starts with a checkout service at a 42 percent error\nrate and ends only after a rollback is durably reconciled and monitoring\nobserves recovery to 3 percent.\n\n```text\nalert\n  -> source-backed observations\n  -> competing hypotheses\n  -> confidence revision\n  -> conflict-preserving resolution\n  -> decision + governing policy\n  -> authorized rollback\n  -> timeout after provider apply\n  -> runtime restart\n  -> provider reconciliation\n  -> verification observation\n  -> resolved incident\n```\n\nThe deterministic run closes and recreates runtime objects twice. The rollback\nis applied once, its first result is `unknown`, and the next runtime reconciles\nthe provider state instead of delivering again.\n\n```json\n{\n  \"finalState\": \"resolved\",\n  \"effectStatus\": \"succeeded\",\n  \"resolutionStatus\": \"resolved_with_conflict\",\n  \"deliveryCount\": 1,\n  \"reconciliationCount\": 1,\n  \"agentRestarts\": 2,\n  \"errorRateBefore\": 0.42,\n  \"errorRateAfter\": 0.03,\n  \"traceNodeCount\": 16,\n  \"traceEdgeCount\": 19\n}\n```\n\nFrom a source checkout, prepare `.env` using the\n[Production Profile](docs/PRODUCTION.md#initial-setup), then run:\n\n```powershell\nnpm install\nnpm run build\ndocker compose up -d postgres\ndocker compose run --rm bootstrap\nnpm run example:sre\n```\n\nSee [Flagship SRE Scenario](docs/SRE_SCENARIO.md) for setup, records, and\nboundaries.\n\n## The causal trace\n\n`explain` traverses explicitly stored lineage around an artifact, assertion,\nworkflow revision, or effect. Effect traces include delivery and\nreconciliation attempts. Artifact plaintext is excluded.\n\nAn abridged rollback trace looks like this. Live output also includes record\nvalues, durable references, and attempt timestamps.\n\n```text\nTrace: effect_...\n\nNodes:\n[0] deployment.rollback effect succeeded\n  attempt 1: unknown\n  attempt 2: succeeded HTTP 200\n  [1] decision remediation\n  [1] directive incident_remediation_policy\n    [2] hypothesis primary_cause\n      [3] observation deployed_version\n      [3] observation database_cpu_change\n      [3] observation error_rate\n  [1] observation error_rate\n\nLinks:\n  assertion:decision --authorizes--> effect_...\n  assertion:policy --governs--> effect_...\n  workflow:incident@3 --produces--> effect_...\n  effect_... --verifies--> assertion:verification\n```\n\nThe same trace is available as bounded structured JSON through TypeScript,\nAgent Intent, HTTP, MCP, and both CLIs. See\n[Explain and Trace](docs/EXPLAIN.md).\n\n## Comparative result\n\nThe repository implements the incident twice:\n\n1. a competent conventional PostgreSQL application with durable attempts,\n   idempotency, reconciliation, lineage, and audit data;\n2. a thin adapter that reuses the shipped Agentic Data Kernel scenario.\n\nCorrectness is required to tie. CI reruns both variants three times and rejects\nstale or inconsistent published evidence.\n\n| Measure | Conventional PostgreSQL | Agentic Data Kernel |\n| --- | ---: | ---: |\n| Correct runs | 3/3 | 3/3 |\n| Delivery count per run | 1 | 1 |\n| Reconciliation count per run | 1 | 1 |\n| Durable audit answers | 9/9 | 9/9 |\n| Observed recovery | 0.42 to 0.03 | 0.42 to 0.03 |\n| Application-owned nonblank lines | 317 | 43 |\n| Application-authored tables | 8 | 0 |\n| Total operated tables | 8 | 18 |\n| Median database footprint | 540,672 bytes | 1,572,864 bytes |\n| Informational median runtime | 50.75 ms | 898.69 ms |\n\nThe result is deliberately not presented as a universal win:\n\n- PostgreSQL matches the kernel on correctness.\n- The smaller adapter reuses 930 lines of shipped scenario code and a 16,424\n  line dependency. It is not an equal from-scratch implementation comparison.\n- The kernel operates more tables, uses more storage, and takes substantially\n  longer in this deterministic smoke run.\n- Runtime is informational because the variants perform different work. This\n  is not a latency benchmark.\n- The test reloads database-backed runtime objects. It does not simulate an\n  operating-system process crash.\n\nRead the [methodology](benchmarks/sre/README.md), the\n[generated report](benchmarks/sre/results/report.md), and the\n[raw evidence](benchmarks/sre/results/summary.json).\n\n## Programming model\n\nThe API is divided into explicit layers while retaining one compatible Agent\nIntent protocol.\n\n| Layer | Responsibility |\n| --- | --- |\n| Knowledge | Entities, immutable evidence, bitemporal assertions, uncertainty, resolution, retrieval, lineage, and explanation |\n| Agency | Guarded workflow revisions, controlled effects, idempotency, attempts, reconciliation, and receipts |\n| Retail compatibility | Existing inventory reservation, payment, and order-expiry behavior |\n\nThe embedded TypeScript API exposes the layers directly:\n\n```ts\nimport { AgenticKernel, SqliteStore } from \"agentic-data-kernel\";\n\nconst store = new SqliteStore(\".data/app.db\");\nconst kernel = new AgenticKernel(store);\n\nconst service = kernel.knowledge.putEntity(\n  {\n    tenantId: \"example\",\n    principalId: \"monitor\",\n    purpose: \"incident-response\"\n  },\n  {\n    entityId: \"service:checkout\",\n    entityType: \"service\",\n    canonicalName: \"Checkout API\"\n  }\n);\n\nconst workflow = kernel.agency.createWorkflow(\n  {\n    tenantId: \"example\",\n    principalId: \"responder\",\n    purpose: \"incident-response\"\n  },\n  {\n    instanceId: \"incident:1001\",\n    workflowType: \"incident_response\",\n    initialState: \"alerted\",\n    data: { service: service.entityId }\n  }\n);\n```\n\nExisting top-level methods and all Agent Intent operation names remain\navailable. TypeScript and HTTP clients submit the same typed envelope, and the\nembedded CLI accepts it from a file:\n\n```json\n{\n  \"protocolVersion\": \"1.0\",\n  \"requestId\": \"observe-1001\",\n  \"idempotencyKey\": \"observe-1001\",\n  \"principal\": {\n    \"tenantId\": \"example\",\n    \"principalId\": \"monitor\",\n    \"purpose\": \"incident-response\"\n  },\n  \"operation\": {\n    \"op\": \"assert\",\n    \"assertion\": {\n      \"subjectEntityId\": \"service:checkout\",\n      \"predicate\": \"error_rate\",\n      \"object\": {\n        \"type\": \"number\",\n        \"value\": 0.42\n      },\n      \"kind\": \"observation\",\n      \"sourceArtifactId\": \"artifact:alert-1001\"\n    }\n  }\n}\n```\n\nMCP tools accept typed operation fields and bind them to the authenticated\nprocess identity. The production CLI provides administration, worker,\nreconciliation, load, and explain commands rather than a general envelope\nexecution command.\n\nSee the [API Reference](docs/API.md).\n\n### Agent middleware\n\nAgentic Data Kernel can be used as lifecycle middleware between an agent host\nand its model. The host receives bounded durable context and JSON Schema tools;\nmodel tool calls are translated into identity-bound Agent Intent operations,\nand approved turns can be recorded as durable artifacts that are encrypted by\nthe production profile.\n\n```ts\nconst session = createEmbeddedAgentMiddleware(kernel, principal).beginRun({\n  runId: \"run:incident-1001\",\n});\n\nconst { context, tools } = await session.prepareModelInput({\n  query: \"why is checkout failing?\",\n  workflow: { instanceId: \"incident:1001\" },\n});\n```\n\nProduction agents can bind an authenticated `ProductionKernel` principal or\nuse the remote HTTP adapter. Direct MCP remains available when the model host\nalready owns context assembly and turn lifecycle. See\n[Agent Middleware](docs/AGENT_MIDDLEWARE.md).\n\nRunnable integrations are included for\n[Microsoft Agent Framework](examples/microsoft-agent-framework) and\n[Azure SRE Agent](examples/azure-sre-agent). The former demonstrates local\nstdio and production Streamable HTTP MCP. The latter configures ADK as a\ndurable incident-context connector and custom SRE specialist.\n\n## When it fits\n\nUse Agentic Data Kernel when several of these are true:\n\n- source disagreement and later correction must remain inspectable;\n- valid time and system time both matter;\n- decisions and external actions need evidence and policy lineage;\n- workflows must resume safely after a restart;\n- provider timeouts can leave an action in an unknown state;\n- tenant, purpose, budget, and effect authority must be enforced below\n  application prompts;\n- agents need bounded, machine-readable causal context.\n\nIt is probably the wrong tool for simple CRUD, append-only analytics, cache\nworkloads, or systems where minimum write latency and minimum schema footprint\nmatter more than durable agency semantics.\n\n## Install and run\n\nThe current stable release is `1.2.0`. Pin an exact version in production.\n\n```powershell\nnpm install agentic-data-kernel@1.2.0\nnpx --yes agentic-data-kernel@1.2.0 example --db .data\\example.db\n```\n\nPackage entry points and commands:\n\n| Interface | Entry point |\n| --- | --- |\n| Embedded TypeScript | `agentic-data-kernel` |\n| PostgreSQL TypeScript | `agentic-data-kernel/production` |\n| Embedded CLI | `agentic-data` or `agentic-data-kernel` |\n| Production CLI | `agentic-data-prod` |\n| MCP | `agentic-data-kernel mcp`, `agentic-data-prod mcp`, or production `POST /mcp` |\n| HTTP | `POST /v1/execute` |\n\nSource checkout:\n\n```powershell\nnpm install\nnpm run build\nnpm test\nnpm run example\n```\n\nIntegration examples cover the TypeScript library, MCP, authenticated HTTP,\nproduction retail, the SRE scenario, embedding providers, and effect\nreceivers. See [Integrations](docs/INTEGRATIONS.md).\n\n## Production profile\n\nThe production profile uses PostgreSQL 18, pgvector, forced row-level\nsecurity, scoped and purpose-bound API keys, encrypted artifact storage,\nconfigurable embedding spaces, effect budgets, authorization fences,\nreconciliation workers, metrics, backups, and Caddy TLS.\n\n```powershell\n.\\scripts\\generate-secrets.ps1\nCopy-Item .env.example .env\n```\n\nReplace every placeholder in `.env` with the generated values, database\npasswords, and embedding-provider configuration before continuing:\n\n```powershell\n$env:AGENTIC_DATA_IMAGE = \"ghcr.io/jason-doyle/agentic-data-kernel:1.2.0\"\ndocker compose --profile server pull\ndocker compose --profile server up --no-build\n```\n\nPostgreSQL remains the transactional storage engine. The kernel supplies the\nagent-facing data model, invariants, and interfaces. See\n[Production Profile](docs/PRODUCTION.md) and\n[Threat Model](docs/THREAT_MODEL.md).\n\n### Cloud deployment templates\n\nThe package includes validated reference workloads for:\n\n| Platform | Template |\n| --- | --- |\n| Kubernetes | Helm |\n| Azure Container Apps | Bicep |\n| AWS ECS Fargate | OpenTofu |\n| Google Kubernetes Engine | OpenTofu plus Helm |\n\nThe templates require existing private PostgreSQL, secret stores, network\ncontrols, TLS, and shared filesystems. They intentionally do not place\ngenerated credentials in Bicep parameters or OpenTofu state. See\n[Deployment Templates](deploy/README.md).\n\n## Architecture\n\n```text\nTypeScript / HTTP / MCP / CLI\n              |\n      Agent Intent validation\n              |\n identity + scope + purpose + budget\n              |\n   +----------+-----------+\n   |                      |\nKnowledge layer       Agency layer\nevidence              workflow revisions\nassertions             authorized effects\nresolution             attempts and receipts\nretrieval              reconciliation\nlineage                verification\n   |                      |\n   +----------+-----------+\n              |\n retail compatibility adapter\n              |\n SQLite development profile\n              or\n PostgreSQL + pgvector + RLS\n```\n\n## Stable support and boundaries\n\n- Version 1.x supports the bounded production profile documented in\n  [Stability and Compatibility](docs/STABILITY.md).\n- Production targets one PostgreSQL primary.\n- The default rate limiter is process-local.\n- One embedding model, version, and dimension is active per deployment.\n- Indexed vector dimensions are limited to 2000.\n- Changing an embedding space after assertions exist requires explicit\n  re-embedding.\n- Generic workflow transition policy remains application-owned.\n- Effect receivers must honor idempotency keys.\n- The SRE telemetry, embeddings, and remediation provider are synthetic.\n- Projection epochs, multi-operation plans, and context-package optimization\n  are not implemented.\n\nSee [Benefits and Tradeoffs](docs/TRADEOFFS.md) for a fuller fit assessment.\n\n## Documentation\n\n- [API reference](docs/API.md)\n- [Flagship SRE scenario](docs/SRE_SCENARIO.md)\n- [Explain and trace](docs/EXPLAIN.md)\n- [SRE benchmark](benchmarks/sre/README.md)\n- [Benefits and tradeoffs](docs/TRADEOFFS.md)\n- [Use cases](docs/USE_CASES.md)\n- [Integration guide](docs/INTEGRATIONS.md)\n- [Production profile](docs/PRODUCTION.md)\n- [Stability and compatibility](docs/STABILITY.md)\n- [Upgrade and rollback](docs/UPGRADING.md)\n- [Production runbooks](docs/RUNBOOKS.md)\n- [Threat model](docs/THREAT_MODEL.md)\n- [Release process](docs/RELEASING.md)\n- [Security policy](SECURITY.md)\n- [Contributing](CONTRIBUTING.md)\n- [Support](SUPPORT.md)\n- [Code of conduct](CODE_OF_CONDUCT.md)\n- [Changelog](CHANGELOG.md)\n\n## License\n\nApache-2.0. See [LICENSE](LICENSE).\n"}