{"id":"60b2a8d6-014b-400c-a53c-56efde845227","slug":"ai-token-optimizer","kind":"mcp","name":"ai-token-optimizer","summary":"Jump-start and measure token-efficient AI coding workflows for GitHub Copilot and Claude Code.","owner":"d2k-klin","repoUrl":"https://github.com/d2k-klin/ai-token-optimizer","license":"MIT","licenseTier":"permissive","trust":"community","stars":6,"match":null,"fewStars":false,"format":"mcp","createdAt":"2026-06-19T10:40:38.000Z","addedAt":"2026-10-21T09:09:55.523Z","logoUrl":"https://avatars.githubusercontent.com/u/71728742?u=efeed5cbdaac32c03f6edb2cbc1654a657078a16&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["developer-tools","search","memory-knowledge"],"tags":["ai","ai-coding","bash","claude-code","cli","context-engineering","developer-tools","github-copilot"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-25T16:53:08.320Z","sample":false,"url":"https://conduits.dev/tools/ai-token-optimizer","language":"Shell","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"

\n \"Mr.\n

\n\n# aito — stop your AI coding agent from burning tokens\n\n[![CI](https://github.com/d2k-klin/ai-token-optimizer/actions/workflows/ci.yml/badge.svg)](https://github.com/d2k-klin/ai-token-optimizer/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n[![shellcheck](https://img.shields.io/badge/shellcheck-clean-brightgreen.svg)](.github/workflows/ci.yml)\n[![bash 3.2+](https://img.shields.io/badge/bash-3.2%2B-green.svg)](#)\n[![aito telemetry: none](https://img.shields.io/badge/aito_telemetry-none-1f6feb.svg)](#privacy--safety)\n[![PRs welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/d2k-klin/ai-token-optimizer/pulls)\n\nGitHub Copilot and Claude Code waste tokens re-reading your repo, dumping build logs into\ncontext, and rebuilding the same explanations every session. **`aito` sets up a\ntoken-efficient workflow in one command — then measures the savings so you don't have to\ntake a number on faith.**\n\nUse it to **jump-start a new project** with AI instructions, persistent context, sensible\ntool defaults, and measurement from the first commit—or add the same workflow safely to\nan established repository. `aito` configures the AI-development layer; your normal\nframework or project generator still creates the application itself.\n\nIt doesn't reinvent anything — it's a **summarized setup of the available tools for token\noptimization during development**: a curated set (OpenWiki, Serena, OpenSpec, RTK,\nccusage, and more) wired together behind an interactive menu with safe defaults. It lays\ndown concise instruction files for each assistant and writes a `token-report.md` you can\nactually read.\n\n```bash\ngit clone https://github.com/d2k-klin/ai-token-optimizer.git\ncd ai-token-optimizer && make install # ~30s, no curl|bash\naito setup # pick tools, get a measured report\n```\n\n\n> 🎬 **Demo:** _(coming — a 15-second `aito setup → verify` recording goes here)_\n\n## Why it's different\n\n- **It measures, it doesn't promise.** No invented \"saves 70%!\" headline — `aito verify`\n reports real token counts and a PASS/WARN verdict you can reproduce.\n- **Retrieval before compression.** Serena, Codebase-Memory-MCP, QMD, and grepai can\n retrieve a symbol, relationship, or document instead of loading whole files first.\n- **No telemetry or proxy in `aito` itself.** It runs offline except the component\n installs you explicitly choose. Headroom is opt-in and off by default; third-party\n telemetry is called out below. See [Privacy & safety](#privacy--safety).\n- **Two tracks:** GitHub Copilot and Claude Code in VS Code — pick one or both.\n- **New or existing projects:** establish the workflow at project creation, or layer it\n onto a mature repository without silently replacing existing configuration.\n- **Safe by construction:** idempotent, never clobbers files (backs up + deep-merges),\n risky options off by default, `shellcheck`-clean with a mocked offline test suite.\n- **Cross-platform:** macOS / Linux, Bash 3.2+ (works with stock macOS bash).\n\n## Tools considered\n\nSelectable tools and documented complements are listed below. See\n[The Tools](docs/tools.md) for the full rundown and\n[Best Results](docs/best-results.md) for how to combine them.\n\n| Tool | What it does | In `aito` |\n|---|---|---|\n| [Caveman](https://github.com/JuliusBrussee/caveman) | Adds concise-output instructions to cut response verbosity (a lite version is always applied). | Default |\n| [Ponytail](https://ponytail.dev) | Ruleset plugin that makes the agent write the least code that works (reuse → stdlib → platform → deps → custom). | Default |\n| [OpenWiki](https://github.com/langchain-ai/openwiki) | Generates and maintains local codebase documentation for coding agents; optional scheduled PR updates. | Default |\n| [OpenSpec](https://github.com/Fission-AI/OpenSpec) | Persistent spec / requirements / design / tasks layer that keeps requirements stable across sessions. | Optional |\n| [Serena](https://github.com/oraios/serena) | Retrieves and edits precise code symbols and references through language servers. | Optional |\n| [Codebase-Memory-MCP](https://github.com/DeusData/codebase-memory-mcp) | Builds a local structural code graph for fast relationship and impact queries. | Optional |\n| [QMD](https://github.com/tobi/qmd) | Runs local BM25 + vector + reranked search over OpenWiki, OpenSpec, and other Markdown. | Optional |\n| [grepai](https://github.com/yoanbernabeu/grepai) | Provides semantic code search and call graphs with local or cloud embeddings. | Optional |\n| [code-review-graph](https://github.com/tirth8205/code-review-graph) | Serves a diff's blast radius over MCP so reviews read affected code, not whole files. | Optional |\n| [Claude-Mem](https://github.com/thedotmack/claude-mem) | Compresses and retrieves agent observations across Claude Code sessions. | Optional (warned) |\n| [RTK](https://github.com/rtk-ai/rtk) | Compresses noisy terminal output (git, tests, builds, logs) before it enters model context. | Optional |\n| [ccusage](https://github.com/ccusage/ccusage) | Local CLI that reports token usage and cost from your agent logs so you can watch the trend. | Optional |\n| [Codesight](https://github.com/Houseofmvps/codesight) | Generates a compact AST-based repo map / wiki so the agent re-reads fewer files. | Optional |\n| [Graphify](https://github.com/Graphify-Labs/graphify) | Maps code plus docs into a knowledge graph for relationship and architecture questions. | Optional |\n| [Repomix](https://github.com/yamadashy/repomix) | Packs the repo into one AI-friendly file with token counts, for one-off exports. | Optional |\n| [gh-aw](https://github.com/github/gh-aw) | Compiles natural-language workflows into GitHub Actions that run AI agents on events. | Optional |\n| [Headroom](https://github.com/headroomlabs-ai/headroom) | Local proxy that compresses context before it reaches the model. | Opt-in (off, warned) |\n| [Context7](https://github.com/upstash/context7) | Fetches current, targeted library/API documentation on demand. | Documented |\n| [code2prompt](https://github.com/mufeedvh/code2prompt) | Packs a codebase into a single prompt with token counts and filtering (Repomix alternative). | Documented |\n| [LLMLingua](https://github.com/microsoft/LLMLingua) | Compresses prompts up to ~20× by dropping low-information tokens (advanced, for custom pipelines). | Documented |\n\nThe layers are intentionally different:\n\n```text\ndon't generate it → Caveman / Ponytail\ndon't retrieve it → Serena / Codebase-Memory-MCP / QMD / grepai / code-review-graph\ndon't rediscover it → OpenWiki / OpenSpec / Claude-Mem / ACE playbook\ncompress when needed → RTK / Headroom / LLMLingua\nmeasure the result → aito verify / ccusage\n```\n\n## Privacy & safety\n\nThis is deliberately boring, which is the point:\n\n- **No network from `aito` itself** except the component installs you pick (npm, PyPI,\n GitHub releases/plugin marketplaces, or an explicitly confirmed upstream installer).\n- **No `aito` telemetry.** Third-party policies still apply. OpenSpec and OpenWiki have\n telemetry enabled upstream; `aito` disables it when it invokes either tool. For manual\n use, set `OPENSPEC_TELEMETRY=0` or `OPENWIKI_TELEMETRY_DISABLED=1` (or\n `DO_NOT_TRACK=1`). Serena's startup metrics use\n `SERENA_USAGE_REPORTING=false`.\n- **Memory/retrieval stays opt-in.** Claude-Mem persists session observations;\n Codebase-Memory, QMD, grepai, and code-review-graph create local indexes; cloud grepai embeddings and\n Context7 queries cross the network. Review the [security model](docs/security.md).\n- **No proxy by default.** The only proxy-based tool (Headroom) is strictly opt-in, off by\n default, and flagged with a warning before install — nothing intercepts your AI traffic\n unless you explicitly choose it.\n- **Non-destructive:** existing files are backed up to `*.bak`; VS Code settings are merged.\n- **Auditable bootstrap:** clone the repo and run its local installer. The optional full\n Caveman install and RTK's non-Homebrew fallback invoke their disclosed upstream\n installers only when selected.\n\n## Documentation\n\n| Guide | What's inside |\n|---|---|\n| **[Getting Started](docs/getting-started.md)** | Prerequisites plus fresh-project and existing-project setup. |\n| **[The Tools](docs/tools.md)** | What each available tool does and why it saves tokens. |\n| **[Best Results](docs/best-results.md)** | Which tools to combine, recipes, and what to avoid. |\n| **[Testing & Proving Token Reduction](docs/testing-token-reduction.md)** | How `aito verify` measures it and how to read the report. |\n| **[Architecture](docs/architecture.md)** | How the CLI is structured and how a run flows. |\n| **[Security model](docs/security.md)** | Per-tool risk ratings and the controls enforced. |\n\nNew here? Start with [Getting Started](docs/getting-started.md).\n\n## Install\n\n```bash\ngit clone https://github.com/d2k-klin/ai-token-optimizer.git\ncd ai-token-optimizer\nmake install # installs `aito` to ~/.local/bin\n# make install PREFIX=/usr/local # system-wide (may prompt for sudo)\n```\n\nPrefer not to use `make`? `bash install.sh` does the same thing\n(`PREFIX=/usr/local bash install.sh` for system-wide).\n\nAdd `~/.local/bin` to your `PATH` if the installer says so. Uninstall with\n`make uninstall` (or `bash install.sh --uninstall`).\n\n## Use\n\n### Jump-start a new project\n\nCreate the application with your usual framework or project generator, then establish\nthe AI workflow before the first AI-assisted task:\n\n```bash\ncd my-new-project\ngit init # skip if the project generator already did this\naito setup\n```\n\nThis gives the project concise assistant instructions, a durable playbook, selected\ntools, and a token-reduction baseline from the start. Initialize OpenWiki after the\nproject has enough source code to document.\n\n### Add it to an existing project\n\nRun from the repository root:\n\n```bash\naito setup # pick track(s) + tools via checkboxes, then auto-verify\naito verify # (re)measure token reduction → token-report.md\naito doctor # check config files, token budgets, and tools\naito learn \"Run rtk tsc before committing\" # add a lesson to the playbook\naito env # show detected environment\n```\n\nNon-interactive (CI or scripted): `AITO_ASSUME_YES=1 aito setup` picks the recommended\ndefaults (Caveman + Ponytail + OpenWiki; everything else, including OpenSpec, RTK, and\nccusage, stays off). The optional OpenWiki documentation-update workflow also stays off.\n\n## What it writes\n\n| Track | Files |\n|---|---|\n| Copilot | `.github/copilot-instructions.md`, `.github/instructions/openspec.instructions.md`, `.vscode/settings.json` |\n| Claude Code | `CLAUDE.md`, `.claude/settings.json`, `.vscode/settings.json` |\n| Shared | `openspec/config.yaml`, `docs/ai-playbook.md` (ACE), `token-report.md` |\n\nExisting files are backed up to `*.bak`; VS Code settings are deep-merged.\nOpenWiki itself writes `openwiki/` plus managed sections in `AGENTS.md` and `CLAUDE.md`\nonly after you run `openwiki --init`. If you approve the separate setup prompt, `aito`\nalso adds `.github/workflows/openwiki-update.yml`.\n\n## How reduction is measured\n\n`aito verify` writes `token-report.md` with four gates: instruction conciseness,\nRTK raw-vs-compressed command output, targeted-vs-whole-repo context, and persistent\nartifact footprint — closed by a PASS/WARN verdict. Uses `tiktoken` when available, else\na labeled chars/4 estimate. See [Testing & Proving Token Reduction](docs/testing-token-reduction.md).\n\n## Configuration (env vars)\n\n| Var | Effect |\n|---|---|\n| `AITO_ASSUME_YES=1` | Non-interactive; accept recommended defaults |\n| `AITO_UI=plain` | Force the plain (read-based) selection UI |\n| `AITO_INSTRUCTION_BUDGET=1500` | Token budget for instruction files |\n| `AITO_OPENWIKI_VERSION` / `AITO_OPENSPEC_VERSION` / `AITO_CCUSAGE_VERSION` | Pin npm component versions |\n| `AITO_SERENA_VERSION` / `AITO_CLAUDE_MEM_VERSION` | Pin Serena or Claude-Mem |\n| `AITO_CODEBASE_MEMORY_VERSION` / `AITO_QMD_VERSION` / `AITO_GREPAI_VERSION` / `AITO_CODE_REVIEW_GRAPH_VERSION` | Pin retrieval components (grepai pin applies to Go builds) |\n| `AITO_RTK_VERSION` | Pin the RTK release installed by its verified upstream installer |\n| `AITO_CAVEMAN_VERSION` | Pin the Caveman release whose installer script is fetched |\n| `AITO_CODESIGHT_VERSION` / `AITO_GRAPHIFY_VERSION` / `AITO_REPOMIX_VERSION` | Pin repository-tool versions |\n| `AITO_HEADROOM_VERSION` | Pin the Headroom Python package version |\n| `NO_COLOR=1` | Disable colored output |\n\n## Development & testing\n\n```bash\nmake test # shellcheck + bats (full local suite); skips a tool if not installed\nmake lint # shellcheck only\nmake unit # bats only\n```\n\nPrereqs: `shellcheck` and `bats` (`brew install shellcheck bats-core` or\n`apt-get install shellcheck bats`). The bats suite mocks all external tools, so it runs\noffline and installs nothing. To try it by hand, run `AITO_ASSUME_YES=1 aito setup`\ninside a throwaway `git init` directory.\n\nMIT licensed. Built by [Mr. D](https://mr-d.dev).\n"}