{"id":"77d8dc6c-4f94-4a27-b75b-a0e5c3e81bfb","slug":"vecscope","kind":"mcp","name":"vecscope","summary":"Deterministic evidence about embedding spaces and vector indexes for agents: hubness, anisotropy, collapse, distance concentration, ANN recall, HNSW/IVF sweeps. CLI + MCP server.","owner":"076923","repoUrl":"https://github.com/076923/vecscope","license":"MIT","licenseTier":"permissive","trust":"community","stars":1,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-10-25T18:26:55.000Z","addedAt":"2026-10-29T09:25:40.421Z","logoUrl":"https://avatars.githubusercontent.com/u/30269358?u=276f78a5af78dd9ee7c0fd75b1a2b99333045306&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["memory-knowledge","search","cloud-infra"],"tags":["agents","embeddings","rag","retrieval","vector-search","vector-scope","approximate-nearest-neighbor","embedding-quality"],"runtime":"python","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-29T10:10:25.216Z","sample":false,"url":"https://conduits.dev/tools/vecscope","language":"Python","transports":["stdio"],"install":{"pypi":"vecscope"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"<!-- mcp-name: io.github.076923/vecscope -->\n\n<div align=\"center\" markdown=\"1\">\n\n<img src=\"https://raw.githubusercontent.com/076923/vecscope/main/images/brand/logo-mark.png\" alt=\"vecscope logo\" width=\"128\">\n\n**English** · [한국어](https://github.com/076923/vecscope/blob/main/docs/ko/README.md)\n\n# vecscope\n\n### Embedding space evidence for agents.\n\n**Before an agent trusts a vector index, let it see the space it is searching.**\n\n[![PyPI version](https://img.shields.io/pypi/v/vecscope?style=flat-square&color=3775a9)](https://pypi.org/project/vecscope/)\n[![Python](https://img.shields.io/pypi/pyversions/vecscope?style=flat-square)](https://pypi.org/project/vecscope/)\n[![PyPI Downloads](https://img.shields.io/pepy/dt/vecscope?style=flat-square&label=downloads&color=2563eb)](https://pepy.tech/projects/vecscope)<br>\n[![CI](https://img.shields.io/github/actions/workflow/status/076923/vecscope/ci.yml?style=flat-square&label=CI)](https://github.com/076923/vecscope/actions/workflows/ci.yml)\n[![Docs](https://img.shields.io/badge/docs-076923.github.io-2563eb?style=flat-square)](https://076923.github.io/vecscope/)\n[![MIT license](https://img.shields.io/badge/license-MIT-f4c430?style=flat-square)](https://github.com/076923/vecscope/blob/main/LICENSE)\n\n[Quick start](#quick-start) · [What it measures](#what-vecscope-measures) · [Use with an agent](#use-with-an-agent) · [Use in CI](#use-in-ci) · [Usage](https://076923.github.io/vecscope/usage/) · [FAQ](#faq) · [Documentation](#documentation)\n\n<code>pip install vecscope</code>\n\n</div>\n\n---\n\nAgents build RAG pipelines, semantic search and recommendation retrieval every day, but they\ncannot see the embedding space they are searching.\n\nWhen retrieval goes wrong it rarely looks like a bug. It looks like:\n\n- \"the same three chunks come back for everything\"\n- \"every score is 0.8-something\"\n- \"top-k is full of copies\"\n- \"brute force finds it and HNSW does not\"\n\nEach of those failures has a known, deterministic diagnostic in the literature. `vecscope`\nturns them into one command and one JSON:\n\n- measurements\n- `ok` / `warn` / `fail` verdicts\n- an explanation an agent can act on\n\nIt runs locally on NumPy, needs no model or network, and reports the same numbers every time.\n\n`vecscope` diagnoses the **vector search implementation**: the geometry the model produced,\nthe hygiene of the corpus, the distance metric, the index and the query pipeline.\n\nIt does not measure relevance. Whether the right documents come back for a question needs\nlabeled queries, and a clean audit is evidence that search is not sabotaging retrieval, not\nproof that retrieval is good.\n\n## Quick start\n\n### 1. Install\n\n```bash\npip install vecscope\n```\n\n### 2. Get a file of embeddings\n\n`vecscope` reads a file with one embedding vector per row: the `(n, dim)` array a model's\n`encode` returns, saved as `.npy`, `.parquet`, `.jsonl`, `.json` or `.csv`\n([all formats](https://076923.github.io/vecscope/usage/#input-formats)).\n\n```python\nimport numpy as np\nembeddings = model.encode(texts)          # shape (n, dim), e.g. (3000, 128)\nnp.save(\"embeddings.npy\", embeddings)\n```\n\nNo embeddings at hand? Generate three synthetic matrices (healthy, anisotropic, collapsed):\n\n```bash\npython -c \"import urllib.request as u; u.urlretrieve('https://raw.githubusercontent.com/076923/vecscope/main/examples/make_examples.py', 'make_examples.py')\"\npython make_examples.py\n```\n\n### 3. Run the audit\n\n```bash\nvecscope audit embeddings.npy\n```\n\n```text\nvecscope x.y.z  n=3000 dim=128 k=10  verdict=FAIL\n  [ok  ] hubness                skewness=1.1661  (ok < 2, warn >= 2, fail >= 4)\n                               k-occurrence skewness 1.17; top 1% of items take 3.9% of all neighbor slots (max N_k=51, expected 10).\n  [WARN] antihubs               antihub_excess=0.0523  (ok < 0.05, warn >= 0.05, fail >= 0.2)\n                               5.2% of items are never anyone's neighbor (random data at k=10 would give 0.0%); the excess is unreachable through k-NN retrieval.\n                               try: inspect a sample of never-retrieved items; drop empty inputs, then apply the hubness fix\n  [FAIL] anisotropy             mean_random_cosine=0.8732  (ok < 0.5, warn >= 0.5, fail >= 0.85)\n                               random pairs have mean cosine 0.873 (isotropic ~0); mean unit vector has norm 0.93; centering would bring the cosine to -0.000.\n                               try: subtract the corpus mean vector and re-normalize, applying the same transform to queries; details.mean_random_cosine_after_centering shows the expected result\n  [ok  ] effective_rank         effective_rank_ratio=0.5407  (ok >= 0.1, warn < 0.1, fail < 0.05)\n                               effective rank 69.2 of 128 attainable (128 dims; 54.1%); low values mean most dimensions carry nothing.\n  [ok  ] collapse               near_duplicate_ratio=0.0000  (ok < 0.02, warn >= 0.02, fail >= 0.1)\n                               0.0% of items have a neighbor with cosine >= 0.99 or an exact copy (exact duplicates 0.0%); no row has a near-duplicate.\n  [ok  ] rogue_dimensions       top1_dim_variance_ratio=1.3070  (ok < 10, warn >= 10, fail >= 50)\n                               dimension 94 alone holds 1.0% of variance, 1.3x the uniform share of 0.78%; top 3 dims hold 3.0%. Rogue dimensions dominate cosine.\n  [ok  ] norms                  norm_cv=0.0000  (ok < 0.1, warn >= 0.1, fail >= 0.3)\n                               vector norms vary with CV 0.000 (median 1.000, range 1.000-1.000); 0.00% are zero vectors. Unequal norms bias inner-product search toward long vectors.\n  [ok  ] metric_mismatch        topk_overlap=1.0000  (ok >= 0.9, warn < 0.9, fail < 0.7)\n                               cosine and inner-product top-10 share 100.0% of neighbors; 100.0% of items get identical lists. Low overlap means the index metric changes the results.\n  [ok  ] distance_concentration contrast_over_noise=2.2784  (ok >= 1.4, warn < 1.4, fail < 1.1)\n                               on 3000 rows a random item is 3.27x farther than the nearest neighbor (random distance 0.127); isotropic noise at 128-d and the same rows would give 1.44x, so the contrast is 2.28x that of noise. On all rows the contrast is 3.27x (median NN distance 0.039). Near 1.0 means distances carry no signal.\n  [ok  ] outliers               outlier_ratio=0.0030  (ok < 0.01, warn >= 0.01, fail >= 0.05)\n                               0.30% of items have no neighbor within cosine distance 0.050 (median 0.039); see details.sample_ids. Typically junk, empty or foreign-language content.\n  [ok  ] connectivity           island_ratio=0.0000  (ok < 0.01, warn >= 0.01, fail >= 0.1)\n                               0.00% of items sit in 0 small island(s) of the k-NN graph (fewer than 15 items; 60 components in total). Graph-based ANN (HNSW) attaches islands to the wrong region or never reaches them.\n```\n\n### 4. Read the result\n\n- The first line gives the overall `verdict`: the worst verdict among the findings.\n- Each finding shows the measured value, the range that would be `ok`, and a sentence saying\n  what the number means for retrieval. Findings that are not `ok` add a `try:` line with the\n  smallest change to test first.\n- The exit code is `1` on `FAIL`, so the same command works as a CI gate.\n- Add `--json` to get the same evidence as JSON for an agent or a script.\n\n## What vecscope measures\n\n| Finding | Metric | ok | warn | fail | What it means when it fails |\n| --- | --- | --- | --- | --- | --- |\n| `hubness` | skewness of the k-occurrence distribution | < 2.0 | 2.0 to 4.0 | ≥ 4.0 | a few items appear in everyone's k-NN list and will top every query |\n| `antihubs` | share of items never retrieved as a neighbor, above the random baseline e^-k | < 5% | 5 to 20% | ≥ 20% | part of the corpus is unreachable through k-NN |\n| `anisotropy` | mean cosine of random pairs | < 0.85 | 0.85 to 0.95 | ≥ 0.95 | vectors crowd into a narrow cone, so scores compress into a band; good retrievers sit well below this without losing nDCG, so only extreme values are flagged |\n| `effective_rank` | participation ratio of the covariance spectrum / attainable rank (dim, or rows − 1 when rows do not exceed dims) | ≥ 0.10 | 0.05 to 0.10 | < 0.05 | most dimensions carry nothing |\n| `collapse` | share of rows with a neighbor at cosine ≥ 0.99 or an exact copy (zero vectors included); details give the duplicate groups and sample pairs | < 2% | 2 to 10% | ≥ 10% | top-k is spent on copies of the same item |\n| `ann_recall` | recall@k of ANN neighbors vs exact k-NN, counted by distance so that exact ties (duplicates) are interchangeable; the tie slack is 1e-6 + 2e-6 x distance, two to three times float32 rounding | ≥ 0.95 | 0.85 to 0.95 | < 0.85 | the index, not the model, is losing results |\n| `rogue_dimensions` | variance of the largest dimension as a multiple of the uniform 1/dim share | < 10x | 10 to 50x | ≥ 50x | a few coordinates decide every cosine score |\n| `norms` | coefficient of variation of vector norms | < 0.10 | 0.10 to 0.30 | ≥ 0.30 | inner-product search favors long vectors; zero vectors are reported too |\n| `metric_mismatch` | top-k overlap between cosine and inner product | ≥ 0.90 | 0.70 to 0.90 | < 0.70 | the index metric changes which results come back |\n| `distance_concentration` | relative contrast (random distance / nearest-neighbor distance) divided by that of isotropic noise at the same dimension and row count | ≥ 1.4 | 1.1 to 1.4 | < 1.1 | the nearest neighbor is barely closer than a random item; 1.0 is pure noise |\n| `outliers` | share of items with an abnormally far nearest neighbor | < 1% | 1 to 5% | ≥ 5% | junk, empty or foreign items; ids are listed in `details` |\n| `connectivity` | share of items in small k-NN graph islands | < 1% | 1 to 10% | ≥ 10% | self-contained groups too small to be clusters; HNSW attaches them badly or never reaches them |\n| `query_gap` | median query-to-corpus nearest distance divided by the median corpus-to-corpus nearest distance | < 2.6 | 2.6 to 3.0 | ≥ 3.0 | queries and corpus do not share a space, typically queries embedded by a different model (5 to 10). Pipelines that reproduce published nDCG@10 sit at 0.94 to 2.52; a missing query prefix does not reliably move it |\n| `query_shift` | share of queries farther from the corpus than the median + 3 MAD of a reference query set (needs a baseline) | < 20% | 20 to 30% | ≥ 30% | the query pipeline moved since the reference: another model or version, or traffic on topics the corpus does not cover. Healthy query sets score 0 to 9% |\n| `ood_queries` | deprecated, read `query_gap`: share of queries farther from the corpus than 99% of corpus items are from each other | < 50% | 50 to 80% | ≥ 80% | same question as `query_gap`, but it counts the normal distance between short queries and long passages too (good models score 0 to 65% on BEIR) and false-alarmed on 4 of 18 reproduced pipelines |\n| `query_hubness` | skewness of corpus k-occurrence over query results | < 4.0 | 4.0 to 10.0 | ≥ 10.0 | the same few items answer most queries; the value tracks nDCG@10 across models (Spearman −0.5 to −0.9), wrong-language models score 11 to 30. It grows with corpus size: good English models score 1.5 to 3.4 up to 26k documents and 4.0 to 4.8 on 58k, so on large corpora compare against a baseline |\n\nEvery finding in the JSON carries its `healthy_range`, `warn_at` and `fail_at`, plus a `details`\nblock with the raw numbers. For `anisotropy` that block includes the cosine the same vectors\nwould have after centering, so the agent can see whether centering alone fixes the problem.\n\nSome findings appear only when you pass their inputs:\n\n- `ann_recall`: an ANN neighbor file with `--ann`\n- `query_gap`, `query_hubness` and `ood_queries`: query embeddings with `--queries`\n- `query_shift`: query embeddings, and `--baseline` also pointing at an earlier audit run with\n  queries\n\nEach finding also carries `likely_causes`, `suggested_action` and `verify_by`. They are\nhypotheses, not prescriptions:\n\n- apply the smallest change\n- re-audit\n- compare retrieval on real queries before adopting it\n\nThe [agent skill](https://github.com/076923/vecscope/blob/main/.agents/skills/vecscope/SKILL.md) spells out\nthat loop.\n\n### Thresholds\n\nThe thresholds are defaults:\n\n- calibrated on [seven public models](https://076923.github.io/vecscope/quality/#real-embeddings)\n- re-checked against 19 models on nine labeled datasets in two languages\n  ([verification](https://github.com/076923/vecscope/blob/main/docs/verification.md))\n\nThey are a starting point for diagnosis, not a universal quality bar. Embedding spaces\ndiffer by model, language, domain and dimensionality, so a `warn` on a good model is normal\nand a clean audit does not guarantee good search.\n\nOverride any pair per run, or derive them from your own data with\n[`vecscope calibrate`](https://076923.github.io/vecscope/quality/#calibrating-thresholds-to-your-domain):\n\n```bash\nvecscope audit vectors.npy --thresholds my_thresholds.json\n# my_thresholds.json: {\"anisotropy.mean_random_cosine\": [0.4, 0.7], \"hubness.skewness\": [2.5, 5]}\n```\n\nThe keys are listed in `src/vecscope/evidence.py` (`THRESHOLDS`) and echoed in every JSON\nresult under `thresholds`.\n\n## Use with an agent\n\n`vecscope` ships an MCP server with four read-only tools: `audit_embeddings`, `sweep_index`,\n`compare_embeddings` and `describe_findings`. Register it with any MCP client:\n\n```bash\npip install vecscope\nclaude mcp add vecscope -- vecscope-mcp --allow-dir /data/embeddings     # Claude Code\n```\n\nThen ask:\n\n```text\nAudit the embeddings in data/chunks.npy before we tune the retriever, and tell me\nwhich finding to fix first.\n```\n\nThe server reads only the directories named with `--allow-dir` (without it, the directory it\nwas started in). [Use with an agent](https://076923.github.io/vecscope/mcp/) has the setup for Cursor, Codex and\nVS Code, the tools and their responses, what the server may read, and the clients it was\ntested with. The [agent skill](https://github.com/076923/vecscope/blob/main/.agents/skills/vecscope/SKILL.md) tells the agent what to do about each verdict.\n\n## Use in CI\n\n`vecscope audit` exits 1 on `FAIL`. Commit a reviewed `--json` run as the baseline, and the\nbuild fails only when a finding gets worse:\n\n```bash\nvecscope audit embeddings.npy --json > baseline.json          # once, reviewed and committed\nvecscope audit embeddings.npy --baseline baseline.json        # on every rebuild\n```\n\nOn GitHub the same gate is an Action:\n\n```yaml\n- uses: 076923/vecscope@v0\n  with:\n    path: embeddings.npy\n    baseline: baseline.json\n    sample: \"0\"\n```\n\n[Use in CI](https://076923.github.io/vecscope/ci/) covers tolerances for slow drift, which rule failed a run, and the\nAction's inputs and outputs.\n\n## FAQ\n\n<details>\n<summary><strong>Does it send my vectors anywhere?</strong></summary>\n\nNo. `vecscope` computes everything with NumPy on your machine. It never loads a model or opens\na network connection.\n\n</details>\n\n<details>\n<summary><strong>How large a matrix can it handle?</strong></summary>\n\nExact k-NN is quadratic in the number of rows. By default `vecscope` audits a seeded random\nsample of 100,000 rows when the input is larger, and says so in the output (`sampled: true`,\n`n_total`).\n\nA sampled result describes the sample, not the whole corpus: rare clusters, small islands and\na handful of outliers can be missed.\n\nRow ids in `details` (`sample_ids`, `sample_pairs`, the cluster report) always refer to rows\nof the input file, also when sampled. Use `--sample 0` to audit everything; see\n[performance](https://076923.github.io/vecscope/performance/#performance) for what that costs.\n\nTo see how much the sample itself moves a number, pass `--sample-repeats 3` (MCP:\n`sample_repeats`). vecscope then audits three independent seeded samples and attaches\n`value_range` (minimum to maximum across samples) and `verdict_stable` to every finding, and\nlists `unstable_findings` at the top level.\n\nA finding whose verdict flips between samples should be read as borderline, not as a property\nof the corpus. The cost is three audits.\n\n`value_range` is the spread between samples, not an interval that contains the full-corpus\nvalue: hubness is a lower bound on every sample, and even a very stable finding can sit a\nlittle outside the range of three samples.\n\nNot every finding means the same thing on a sample: hubness is understated, outliers\novercounted, islands mostly invisible. The [performance](https://076923.github.io/vecscope/performance/#performance) table says what each\nfinding can stand for and which verdicts a sampled audit marks `inconclusive`.\n\n</details>\n\n<details>\n<summary><strong>My production model fails the anisotropy check. Is the model broken?</strong></summary>\n\nNot necessarily. Many popular text embedding models are strongly anisotropic by construction.\n\nThe finding tells you that raw cosine scores are compressed into a narrow band; centering or\nwhitening the vectors, or re-ranking, usually restores discrimination. The `details` block\nreports the cosine after centering so you can see whether that is enough. The verdict is\nevidence for the agent, not a grade for the model.\n\n</details>\n\n<details>\n<summary><strong>Which agents can use it?</strong></summary>\n\nAny agent that supports local MCP servers, and any agent that can run a shell command and read\nJSON. The skill file describes both paths.\n\n</details>\n\n## Documentation\n\n- [Documentation site](https://076923.github.io/vecscope/): this README and the pages below, searchable\n- [Usage](https://076923.github.io/vecscope/usage/): queries and clusters, the index sweep, comparing two embeddings,\n  vector stores, incremental audits, the library, CLI options and input formats\n- [Use with an agent](https://076923.github.io/vecscope/mcp/): MCP setup per client, tools and responses, what the\n  server may read, tested clients\n- [Use in CI](https://076923.github.io/vecscope/ci/): baselines, tolerances, the gate and the GitHub Action\n- [Retrieval quality and thresholds](https://076923.github.io/vecscope/quality/): the fix order, real embeddings, whether\n  the findings predict nDCG, calibrating thresholds to your domain\n- [Performance](https://076923.github.io/vecscope/performance/): audit time and memory, sampled audits, how vecscope works\n- [Agent skill](https://github.com/076923/vecscope/blob/main/.agents/skills/vecscope/SKILL.md): when to run the audit and what to do with each verdict\n- [Retrieval-quality tables](https://github.com/076923/vecscope/blob/main/docs/retrieval_quality.md): every finding against nDCG@10 on labeled data\n- [Verification](https://github.com/076923/vecscope/blob/main/docs/verification.md): exactness, recall scoring, the sampling claims, the query findings, interventions, agent integration, and 19 models on 9 labeled datasets\n- [Agent cases](https://github.com/076923/vecscope/blob/main/docs/agent_case.md): an agent finding and fixing two ingestion faults, and a mis-tuned IVF index, over MCP, with the tool calls and numbers\n- [Examples](https://github.com/076923/vecscope/tree/main/examples/): synthetic matrices, the calibration script and the retrieval-quality study\n- [Changelog](https://github.com/076923/vecscope/blob/main/CHANGELOG.md)\n\n## Contributing\n\nIssues and pull requests are welcome. See [CONTRIBUTING.md](https://github.com/076923/vecscope/blob/main/CONTRIBUTING.md) for how to run\nthe tests and what a good change looks like. Security reports go through\n[SECURITY.md](https://github.com/076923/vecscope/blob/main/SECURITY.md).\n\n## License\n\n[MIT](https://github.com/076923/vecscope/blob/main/LICENSE)\n"}