![\"TokensCache\"](\"assets/banner.svg\")
\n\n \n
\n ![\"TokensCache](\"assets/logo.svg\"){kind=logo}
\n
\n AI token waste elimination & an intelligent cache layer for multi-provider agents.\n
\n\n\n exact match → semantic reuse → optimize → budget → provider\n
\n ![\"CI\"](\"https://img.shields.io/github/actions/workflow/status/kais-aljammal/tokenscache/ci.yml?style=for-the-badge&label=CI&labelColor=0b1218\"/)
![\"version\"](\"https://img.shields.io/badge/v1.0.0-stable-2dd4bf?style=for-the-badge&labelColor=0b1218\"/){kind=icon}
![\"license\"](\"https://img.shields.io/badge/license-MIT-5eead4?style=for-the-badge&labelColor=0b1218\"/){kind=icon}
![\"node\"](\"https://img.shields.io/badge/node-%3E%3D20-34d399?style=for-the-badge&labelColor=0b1218\"/){kind=icon}
\n Quick start ·\n Results ·\n Architecture ·\n Config ·\n Author\n
\n\n---\n\n```text\n ┌────────────── TokensCache ──────────────┐\n │ │\n │ ◆ L1 memory ◆ L2 IndexedDB │\n │ ◆ L3 SQLite ◆ semantic ANN │\n │ │\n │ prune · compress · shape · enforce $ │\n │ │\n │ OpenAI · Anthropic · Gemini │\n └──────────────────────────────────────────┘\n fewer calls · same output\n```\n\n## Why it exists\n\nAgents repeat themselves. Same plans. Same tool dumps. Same boilerplate.\n\n**TokensCache** sits in front of your LLM calls and:\n\n- caches exact + semantic matches (L1 → L2 → L3)\n- prunes / compresses / shapes prompts before they burn tokens\n- enforces real-dollar budgets with warn / downgrade / block\n- keeps output quality — verified in mock agent A/B tests\n\n## Status\n\n**v1.0.0** — cache stack, optimizer, budget enforcement, provider adapters, MCP / proxy / dashboard tooling.\n\n## Quick start\n\n```bash\ngit clone https://github.com/kais-aljammal/tokenscache.git\ncd tokenscache\nnpm install\nnpm test\nnpm run build\n```\n\nCopy `.env.example` to `.env` only if you are calling real providers — never commit that file.\n\nThe package is not on npm yet — clone and use it from source (or `npm pack` / a local `file:` install after `npm run build`).\n\n### Minimal Node usage\n\n```typescript\nimport { TokensCache } from \"tokenscache\";\n\nconst cache = new TokensCache({\n config: {\n providers: { openai: { apiKey: process.env.OPENAI_API_KEY } },\n cache: { l1: { maxEntries: 500 } },\n },\n});\n\n// Register a provider adapter, then:\n// const response = await cache.chat({\n// provider: \"openai\",\n// model: \"gpt-4o-mini\",\n// messages: [...],\n// });\n```\n\nRun the mock demo (no API keys):\n\n```bash\nnpx tsx examples/node-agent/index.ts\n```\n\n### Tooling\n\n```bash\nnpm run bench # Cache hit-rate benchmark (200 prompts, 30% semantic overlap)\nnpm run audit # Token waste report from ./tokenscache.db\nnpm run sync-pricing # Verify config/pricing.json is ≤30 days old\nnpm run cashier-compare # A/B demo: agent with vs without TokensCache\n```\n\nIntegration tests in `tests/integration/agent-tasks.test.ts` run 10 mock coding-agent scenarios and assert token savings with identical output.\n\n## Test results\n\nAll numbers below are from the repo's automated test suite — no API keys required. Reproduce anytime with `npm test` and `npm run cashier-compare`.\n\n### Test suite\n\n| Check | Result |\n|-------|--------|\n| Unit + integration tests | **139 / 139 passing** |\n| TypeScript | zero errors (`npm run typecheck`) |\n| Build | ESM + DTS (`npm run build`) |\n\n### 10 coding-agent scenarios (integration tests)\n\nMock agents build real TypeScript modules (todo API, URL shortener, auth JWT, etc.) across **105 agent turns**. TokensCache uses `agentArtifactScope` + `verified-decision` semantic matching.\n\n| Metric | Without TokensCache | With TokensCache | Savings |\n|--------|-------------------:|----------------:|--------:|\n| Upstream LLM calls | 105 | 66 | **37%** |\n| Total tokens billed | 42,066 | 27,136 | **35.5%** |\n| Cache hits | 0 | 39 | — |\n| Output identical | — | **yes** (all 10 tasks) | — |\n\nPer-task token savings range from **28%** to **39%**. Every scenario asserts identical generated code with vs without TokensCache.\n\n### Cashier A/B demo (`npm run cashier-compare`)\n\nSingle-task side-by-side: 15-turn agent building a TypeScript POS system.\n\n| Metric | Without | With | Savings |\n|--------|--------:|-----:|--------:|\n| Upstream LLM calls | 15 | 9 | 40% |\n| Total tokens | 6,382 | 3,893 | **39.0%** |\n| Quality score | 9.8/10 | 9.8/10 | unchanged |\n| Output identical | — | **yes** | — |\n\n```text\n without ████████████████████ 6,382 tokens\n with ████████████░░░░░░░░ 3,893 tokens (−39%)\n```\n\n## Architecture\n\n```text\n Request\n │\n ▼\n ┌─────────────┐\n │ CacheRouter │── L1 (memory) → L2 (IndexedDB) → L3 (SQLite) → Semantic\n └──────┬──────┘\n │ miss\n ▼\n ┌─────────────┐\n │ Optimizer │── tool prune · history compress · output shape · cache align\n └──────┬──────┘\n ▼\n ┌──────────────┐\n │BudgetEnforcer│── estimate cost · check limits · downgrade route\n └──────┬───────┘\n ▼\n ┌─────────────┐\n │ Provider │── OpenAI · Anthropic · Gemini\n └─────────────┘\n```\n\n| Layer | Module | Description |\n|-------|--------|-------------|\n| **L1** | `l1-memory.ts` | In-memory LRU exact-match cache |\n| **L2** | `l2-browser.ts` | Browser IndexedDB persistence |\n| **L3** | `l3-local.ts` | SQLite + embedding ANN search |\n| **Semantic** | `semantic-match.ts` | Cosine similarity with threshold policies |\n| **Optimizer** | `optimizer/*` | Token reduction before provider calls |\n| **Budget** | `budget/*` | Real-dollar ledger and enforcement |\n| **Providers** | `providers/*` | Unified adapter interface |\n\nBrowser apps import from `tokenscache/browser` — see [examples/browser-react/README.md](examples/browser-react/README.md).\n\nMCP guide: [examples/mcp-server/README.md](examples/mcp-server/README.md) · Agent demo: [examples/cashier-comparison/README.md](examples/cashier-comparison/README.md).\n\n## Configuration reference\n\nConfiguration is validated with Zod via `TokensCacheConfigSchema`.\n\n```typescript\n{\n providers: {\n openai: { apiKey?: string, baseUrl?: string },\n anthropic: { apiKey?: string, baseUrl?: string },\n google: { apiKey?: string, baseUrl?: string },\n },\n budget?: {\n daily?: { usd: number, action: \"warn\" | \"downgrade\" | \"block\" },\n session?: { usd: number, action: \"warn\" | \"downgrade\" | \"block\" },\n hard?: { usd: number, action: \"warn\" | \"downgrade\" | \"block\" },\n includeCacheStorageHoldingCosts?: boolean, // default true\n },\n cache?: {\n l1?: { maxEntries: number, ttlMs?: number }, // default 500\n l2?: { dbName: string, maxSizeMB: number }, // browser\n l3?: { dbPath: string, maxSizeMB: number, maxEntries?: number },\n semantic?: {\n highThreshold: number, // default 0.92\n grayZoneMin: number, // default 0.7\n matchPolicy: \"static-threshold\" | \"verified-decision\",\n maxCandidates?: number,\n },\n includeModelInKey?: boolean, // default true — gpt-4o ≠ gpt-4o-mini\n includeToolsInKey?: boolean, // default true\n agentArtifactScope?: boolean, // reuse cache per metadata.artifact\n gemini?: { explicitCacheSafetyMargin: number },\n },\n optimizer?: {\n toolPruning: boolean,\n historyCompression: boolean,\n compressionTrigger: number,\n outputShaping: boolean,\n outputShapingTrigger: number,\n outputShapingHoldout: number,\n cacheAlignment: boolean,\n },\n}\n```\n\nPricing tiers live in [config/pricing.json](config/pricing.json). Refresh when providers change rates:\n\n```bash\nnpm run sync-pricing\n```\n\n## Contributing & security\n\n- [CONTRIBUTING.md](CONTRIBUTING.md)\n- [SECURITY.md](SECURITY.md)\n- [License audit](docs/license-audit.md)\n\n## License\n\nTokensCache is released under the **[MIT License](LICENSE)**.\n\nYou may use, modify, and distribute this software freely, including in commercial projects, as long as the license notice is included.\n\nThird-party inspiration and dependency licenses are documented in [docs/license-audit.md](docs/license-audit.md).\n\n---\n\n\n\n\n \n
\n \n ✦ crafted by Kais Aljammal · kaiss.tech
\n Instagram → @kaiss.tech
\n GitHub → @kais-aljammal\n \n