{"id":"920dd18b-251d-4422-b771-23709c5eb266","slug":"markdown-matters","kind":"mcp","name":"markdown-matters","summary":"Structural markdown intelligence for LLMs — search, index, and summarize with 80% fewer tokens","owner":"srobinson","repoUrl":"https://github.com/srobinson/markdown-matters","license":"MIT","licenseTier":"permissive","trust":"community","stars":3,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-02-21T13:47:57.000Z","addedAt":"2026-10-21T08:23:58.825Z","logoUrl":"https://avatars.githubusercontent.com/u/261612?u=ef1ed03d62834d0369a3c53d168f2d08889189d7&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["files-documents","search","writing"],"tags":["ai-tools","cli","context-window","documentation","embeddings","llm","markdown","semantic-search"],"runtime":"node","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-25T16:33:17.438Z","sample":false,"url":"https://conduits.dev/tools/markdown-matters","language":"TypeScript","transports":["stdio"],"install":{"npm":"markdown-matters"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# markdown-matters\n\n**Give LLMs exactly the markdown they need. Nothing more.**\n\n```bash\nQUICK REFERENCE\n  mdm init [options]              Initialize mdm in a directory\n  mdm index [path] [options]      Refresh the active manifest (add --embed for semantic search)\n  mdm fix [path] [options]        Repair malformed YAML frontmatter\n  mdm search <query> [options]    Search by meaning or structure\n  mdm context <files...>          Get LLM-ready summary\n  mdm tree [path]                 Show files or document outline\n  mdm config <command>            Configuration management (init, show, check)\n  mdm duplicates [path]           Find duplicate content\n  mdm embeddings <command>        Manage embedding namespaces\n  mdm links <file>                Outgoing links\n  mdm backlinks <file>            Incoming links\n  mdm stats [path]                Index statistics\n```\n\n---\n\n## Why?\n\nYour documentation is 50K tokens of markdown. LLM context windows are limited. Raw markdown dumps waste tokens on structure, headers, and noise.\n\nmdm extracts *structure* instead of dumping *text*. The result: **80%+ fewer tokens** while preserving everything needed to understand your docs.\n\nmdm indexes Obsidian, Foam, and Logseq style `[[wikilinks]]` alongside standard Markdown links.\n\n```bash\nnpm install -g markdown-matters\nmdm index .                     # Append this path and refresh the manifest\nmdm fix .                       # Preview malformed frontmatter repairs\nmdm search \"authentication\"     # Find by meaning\nmdm context README.md           # Get LLM-ready summary\n```\n\n---\n\n## Installation\n\n```bash\nnpm install -g markdown-matters\n```\n\nRequires Node.js 18+. Semantic search requires an embedding provider (OpenAI, Ollama, LM Studio, OpenRouter, or Voyage). See [docs/CONFIG.md](./docs/CONFIG.md#embedding-providers) for provider setup.\n\n---\n\n## Commands\n\n### init\n\nInitialize mdm in a directory. Supports both local project setup and global shared indexing.\n\n```bash\nmdm init                        # Interactive setup (prompts for local or global)\nmdm init --local                # Create only .mdm.toml in the current directory\nmdm init --global               # Initialize the active MDM_HOME (default: ~/.mdm)\nmdm init --yes                  # Accept all defaults without prompting\n```\n\nLocal setup creates only `.mdm.toml`; it does not create an index directory or add the current directory to the manifest. Global setup creates the active home, using `~/.mdm` by default or the `MDM_HOME` override, and appends the current directory to `manifest.toml`.\n\nConfig files merge by key in this order: active home `.mdm.toml`, project `.mdm.toml`, then project `.mdm.local.toml`. Environment variables and CLI flags have higher precedence.\n\n### index\n\nRefresh the active manifest for fast searching.\n\n```bash\nmdm index .                     # First index: append cwd, then refresh the manifest\nmdm index                       # Later: refresh all existing manifest directories\nmdm index ./docs                # Append path, then refresh all directories\nmdm index . --embed             # First index and build semantic embeddings\nmdm index --no-embed            # Leave semantic vectors unchanged\nmdm index --force-embed         # Rebuild every semantic embedding\nmdm index --watch               # Fails; multi-root manifest watch is unavailable\nmdm index --force               # Bypass cache, re-process all files\nmdm index --exclude \"*.draft.md,research/**\"  # Exclude patterns (comma-separated)\nmdm index --no-gitignore        # Ignore .gitignore file\n```\n\nWith a path, `mdm index` appends its absolute declared path to `manifest.toml` before refreshing every directory. Without a path, it refreshes the existing manifest and fails with guidance when the manifest is empty. After `mdm init --local`, the first index therefore needs a path such as `mdm index .`. mdm respects `.gitignore` and `.mdmignore` patterns. Use `--exclude` to add CLI-level patterns. `--watch` fails with a `ManifestError` until multi-root manifest watching is available.\n\n### fix\n\nRepair malformed YAML frontmatter in markdown files.\n\n```bash\nmdm fix                         # Preview repairs in current directory\nmdm fix ./docs                  # Preview repairs in a directory\nmdm fix docs/broken.md          # Preview repairs for one file\nmdm fix ./docs --write          # Apply repairs\nmdm fix ./docs --write --force  # Apply even when tracked files are modified\nmdm fix ./docs --json           # Output structured report\n```\n\nBy default, `mdm fix` is a dry run. It lists files that would change and shows line-level `-` and `+` diffs for repaired frontmatter lines. Use `--write` to apply repairs.\n\nWhen writing, mdm skips tracked files with uncommitted modifications and prints `skipped (uncommitted changes): <path>`. Untracked files, clean tracked files, and files outside git repositories proceed normally. Use `--force` to bypass the dirty-file guard.\n\n### search\n\nSearch by meaning (semantic) or keyword (text match).\n\n```bash\nmdm search \"how to authenticate\"        # Semantic search (if embeddings exist)\nmdm search -k \"auth.*flow\"              # Keyword search (text match)\nmdm search -n 5 \"setup\"                 # Limit to 5 results\nmdm search --threshold 0.25 \"deploy\"    # Lower threshold for more results\n```\n\n#### Similarity Threshold\n\nSemantic search filters results by similarity score (0-1). Default: **0.35** (35%).\n\n- **0 results?** Content may exist below the threshold. Try `--threshold 0.25`\n- **Typical scores**: Single-word queries score ~30-40%, multi-word phrases ~50-70%\n- **Higher threshold** = stricter matching, fewer results\n- **Lower threshold** = more results, possibly less relevant\n\n```bash\nmdm search \"authentication\"              # Uses default 0.35 threshold\nmdm search --threshold 0.25 \"auth\"       # Lower threshold for broad queries\nmdm search --threshold 0.6 \"specific\"    # Higher threshold for precision\n```\n\n#### Context Lines\n\nShow surrounding lines around matches (like grep):\n\n```bash\nmdm search \"checkpoint\" -C 3            # 3 lines before AND after each match\nmdm search \"error\" -B 2 -A 5            # 2 lines before, 5 lines after\n```\n\nAuto-detection: Uses semantic search if embeddings exist and query looks like natural language. Use `-k` to force keyword search.\n\n#### Advanced Search\n\n**Quality Modes** - Control speed vs. accuracy tradeoff:\n```bash\nmdm search \"query\" --quality fast       # 40% faster, good recall\nmdm search \"query\" -q thorough          # Best recall, 30% slower\n```\n\n**Re-ranking** - Boost precision by 20-35%:\n```bash\nmdm search \"query\" --rerank             # First use downloads 90MB model\nnpm install @huggingface/transformers         # Required dependency\n```\n\n**HyDE** - Better results for complex questions:\n```bash\nmdm search \"how to implement auth\" --hyde   # Expands query semantically\n```\n\n#### AI Summarization\n\nGenerate AI summaries of search results with an installed, authenticated Claude Code CLI:\n\n```bash\nmdm search \"authentication\" --summarize     # Get AI summary of results\nmdm search \"database\" -s --stream           # Forward output chunks as received\n```\n\nSummarization currently uses `claude -p` with Claude Code's active authentication, including supported subscriptions. Other CLI and API summarization providers are not yet implemented. See [AI Summarization](#ai-summarization) for setup.\n\n### context\n\nGet LLM-ready summaries from one or more files.\n\n```bash\nmdm context README.md                   # Single file\nmdm context README.md docs/api.md       # Multiple files\nmdm context docs/*.md                   # Glob patterns work\nmdm context -t 500 README.md            # Token budget\nmdm context --brief README.md           # Minimal output\nmdm context --full README.md            # Include full content\n```\n\n#### Section Filtering\n\nExtract specific sections instead of entire files:\n\n```bash\nmdm context doc.md --sections           # List available sections\nmdm context doc.md --section \"Setup\"    # Extract by section name\nmdm context doc.md --section \"2.1\"      # Extract by section number\nmdm context doc.md --section \"API*\"     # Glob pattern matching\nmdm context doc.md --section \"Config\" --shallow  # Top-level only (no nested subsections)\n```\n\nThe `--sections` flag shows all sections with their numbers and token counts, helping you target exactly what you need.\n\n### tree\n\nShow file structure or document outline.\n\n```bash\nmdm tree                        # List markdown files in current directory\nmdm tree ./docs                 # List files in specific directory\nmdm tree README.md              # Show document outline (heading hierarchy)\n```\n\nAuto-detection: Directory shows file list, file shows document outline.\n\n### links / backlinks\n\nAnalyze link relationships.\n\nmdm indexes standard Markdown links such as `[Guide](./guide.md)` and these wikilink forms:\n\n- `[[Note]]` links to a note.\n- `[[Note|alias]]` uses the alias for display only.\n- `[[Note#Heading]]` records a section edge.\n- `[[folder/Note]]` targets a path.\n\nPath shaped targets resolve exactly. Other targets resolve by basename, case insensitively, across the indexed corpus. Unresolved targets are reported as broken links and do not create phantom edges.\n\n```bash\nmdm links README.md             # What does this file link to?\nmdm backlinks docs/api.md       # What files link to this?\n```\n\n### stats\n\nShow index statistics.\n\n```bash\nmdm stats                       # Current directory\nmdm stats ./docs                # Specific path\n```\n\n### duplicates\n\nDetect duplicate content in markdown files.\n\n```bash\nmdm duplicates                  # Find duplicates in current directory\nmdm duplicates docs/            # Find duplicates in specific directory\nmdm duplicates --min-length 100 # Only flag sections over 100 characters\nmdm duplicates -p \"docs/**\"     # Filter by path pattern\n```\n\n### embeddings\n\nManage embedding providers and namespaces.\n\n```bash\nmdm embeddings list             # List all embedding namespaces\nmdm embeddings current          # Show active namespace\nmdm embeddings switch openai    # Switch to OpenAI embeddings\nmdm embeddings remove ollama    # Remove Ollama embeddings\nmdm embeddings remove openai -f # Force remove active namespace\n```\n\nNamespaces store embeddings separately by provider/model. Switching is instant without rebuild.\n\n---\n\n## Workflows\n\n### Before Adding Context to LLM\n\n```bash\nmdm tree docs/                          # See what's available\nmdm tree docs/api.md                    # Check document structure\nmdm context -t 500 docs/api.md          # Get summary within token budget\n```\n\n### Finding Documentation\n\n```bash\nmdm search \"authentication\"             # By meaning\nmdm search -k \"Setup|Install\"           # By keyword pattern\n```\n\n### Setting Up Semantic Search\n\nmdm supports multiple embedding providers for semantic search:\n\n- **OpenAI** (default) - Cloud-based, requires API key\n- **Ollama** - Free, local, daemon-based\n- **LM Studio** - Free, local, GUI-based (development only)\n- **OpenRouter** - Multi-provider gateway\n- **Voyage** - Premium quality, competitive pricing\n\nQuick start with OpenAI:\n```bash\nexport OPENAI_API_KEY=sk-...\nmdm index . --embed                     # First index and build embeddings\nmdm search \"how to deploy\"              # Now works semantically\n```\n\nUsing Ollama (free, local):\n```bash\nollama serve && ollama pull nomic-embed-text\nmdm index . --embed --provider ollama --provider-model nomic-embed-text\n```\n\nSee [docs/CONFIG.md](./docs/CONFIG.md#embedding-providers) for complete provider setup, comparison, and configuration options.\n\n---\n\n## MCP Integration\n\nFor Claude Desktop, add to `~/Library/Application Support/Claude/claude_desktop_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"mdm\": {\n      \"command\": \"mdm-mcp\",\n      \"args\": []\n    }\n  }\n}\n```\n\nFor Claude Code, add to `.claude/settings.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"mdm\": {\n      \"command\": \"mdm-mcp\",\n      \"args\": []\n    }\n  }\n}\n```\n\n### MCP Tools\n\n| Tool | Description |\n|------|-------------|\n| `md_search` | Semantic search by meaning; returns relevant sections |\n| `md_context` | Token-compressed file summaries at `brief`, `summary`, or `full` detail |\n| `md_structure` | Heading hierarchy with token counts |\n| `md_keyword_search` | Structural search by heading, code, list, or table presence |\n| `md_index` | Build or rebuild the index |\n| `md_links` | Outgoing links from a file |\n| `md_backlinks` | Incoming links to a file |\n\n---\n\n## Configuration\n\nmdm supports a layered configuration system for persistent settings:\n\n```bash\n# Create a config file\nmdm config init\n\n# Check your configuration\nmdm config check\n\n# Customize settings in .mdm.toml\n```\n\n```toml\n# .mdm.toml\n[index]\nmaxDepth = 10\nexcludePatterns = [\"node_modules\", \".git\", \"dist\", \"build\"]\n\n[search]\ndefaultLimit = 20\nminSimilarity = 0.35\n```\n\nConfiguration precedence: CLI flags > Environment variables > Config file > Defaults\n\n**See [docs/CONFIG.md](./docs/CONFIG.md) for the complete configuration reference.**\n\n### Index Location\n\nThe database lives in the active `MDM_HOME`, which defaults to `~/.mdm` and can be overridden with the `MDM_HOME` environment variable. Projects do not own separate index trees.\n\n```\n$MDM_HOME/\n  .mdm.toml           # Optional home configuration\n  manifest.toml       # Declared source directories\n  current             # Pointer to the active gen-N\n  staging/            # Unpublished build workspace\n  gen-N/              # Immutable published generation\n    indexes/          # Documents, sections, links, and BM25 data\n    embeddings/       # Semantic vectors when enabled\n    leases/           # Active reader leases\n```\n\nEach refresh builds and validates a complete generation under `staging/`, renames it to `gen-N`, then atomically replaces the `current` pointer. Readers keep using one complete generation throughout an operation. Older generations remain intact until active reader leases are released and cleanup can remove them.\n\n### Environment Variables\n\n| Variable | Description |\n|----------|-------------|\n| `MDM_HOME` | Database home containing `manifest.toml` and generations (default: `~/.mdm`) |\n| `OPENAI_API_KEY` | Required for OpenAI semantic search (default provider) |\n| `OPENROUTER_API_KEY` | Required for OpenRouter semantic search |\n| `MDM_*` | Configuration overrides (see [CONFIG.md](./docs/CONFIG.md)) |\n\n---\n\n## AI Summarization\n\nTransform search results into a Claude Code summary.\n\n### Quick Start\n\n```bash\nmdm search \"authentication\" --summarize\nmdm search \"database\" --summarize --stream\n```\n\n### Setup\n\nInstall Claude Code and authenticate it before using `--summarize`. mdm invokes `claude -p` and uses Claude Code's active authentication. Claude Pro, Max, Team, and Enterprise subscriptions can authenticate through Claude Code. If `ANTHROPIC_API_KEY` is set, Claude Code applies its own credential precedence.\n\nmdm reports the selected provider before the summary:\n\n```\nUsing claude (subscription - FREE)\n\n--- AI Summary ---\n\nBased on the search results, here are the key findings...\n```\n\n### Current Provider\n\n| Provider | Command | Status |\n|----------|---------|--------|\n| Claude Code | `claude` | Supported |\n| Other CLI and API providers | | Not yet implemented |\n\n### Configuration\n\n```toml\n# .mdm.toml\n[aiSummarization]\nmode = \"cli\"\nprovider = \"claude\"\n```\n\n### CLI Flags\n\n| Flag | Short | Description |\n|------|-------|-------------|\n| `--summarize` | `-s` | Enable AI summarization |\n| `--stream` | | Forward Claude CLI output chunks as received |\n\n---\n\n## Performance\n\n| Metric | Raw Markdown | mdm | Savings |\n|--------|--------------|---------|---------|\n| Context for single doc | 2,500 tokens | 400 tokens | **84%** |\n| Context for 10 docs | 25,000 tokens | 4,000 tokens | **84%** |\n| Search latency | N/A | <100ms | - |\n\n---\n\n## License\n\nMIT\n"}