{"id":"97146a05-06d4-43ed-84cb-f1e79686c571","slug":"catnip","kind":"skill","name":"catnip","summary":"Capture, analyze, visualize, and retain GitHub repo metrics locally with stdlib-only Python.","owner":"TGPSKI","repoUrl":"https://github.com/TGPSKI/catnip","license":"GPL-3.0","licenseTier":"strong-copyleft","trust":"community","stars":5,"match":null,"fewStars":false,"format":"skill","createdAt":"2026-08-07T05:36:40.000Z","addedAt":"2026-10-21T11:21:05.907Z","logoUrl":"https://avatars.githubusercontent.com/u/12532803?u=1cb01632caabd7d7b83deb0a24582054303d6f56&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["developer-tools","data-analytics"],"tags":["analytics","cli","curses","github-api","github-traffic","python","self-hosted","stdlib"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-26T03:06:17.294Z","sample":false,"url":"https://conduits.dev/tools/catnip","language":"Python","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# catnip\n\n[changelog](CHANGELOG.md) | [metrics & data layout](docs/metrics.md) | [configuration](docs/configuration.md) | [automation](docs/automation.md) | [pate.sh](https://pate.sh)\n\n**Capture, analyze, visualize, and retain GitHub repo metrics locally with stdlib-only Python.**\n\nClones, views, unique cloners, popular paths, referrers, stars and forks with\ntimestamps, releases, pull requests, languages, commit activity — every repo\nyou own or administer, collected daily and read in your terminal. Traffic,\npopular paths, referrers and release/push events go into a permanent store\nthat is never pruned.\n\nPython 3.10+ (stdlib only) and the [GitHub CLI](https://cli.github.com). No\nservice, no build step, no data leaving your machine.\n\n```bash\n# Prereqs\ngh auth login --scopes repo     # traffic endpoints need push access\ngit clone git@github.com:TGPSKI/catnip.git\ncd catnip\nmake install\n\n# Agent quickstart\nmake link-agents\nclaude \"/catnip-onboarding\"\nopencode run \"/catnip-onboarding\"\n\n# CLI quickstart\ncatnip init                     # write a config (or --owner X --no-input)\ncatnip run                      # fetch -> analyze -> history -> totals -> verify\n\n# Visualize + Analyze\ncatnip tui                      # terminal data visualization\ncatnip report                   # deterministic analysis\n\n# Automate\ncatnip timer install            # automate collection with systemd\ncatnip timer cron               # automate collection with cron\n```\n\n## Agent Integrations\n\n`catnip` comes with skills for onboarding, troubleshooting, and analysis:\n\n| skill | for |\n|---|---|\n| [**`catnip-onboarding`**](.agents/skills/catnip-onboarding/SKILL.md) | guided install from fresh clone to full automation |\n| [**`catnip-triage`**](.agents/skills/catnip-triage/SKILL.md) | guided troubleshooting |\n| [**`catnip-prowl`**](.agents/skills/catnip-prowl/SKILL.md) | deterministic analysis and agentic inference  |\n\nIntegrate with your harness:\n\n```bash\nmake link-agents                          # .claude, .cursor, .opencode\nmake link-agents HARNESSES=\".aider\"       # or wherever yours looks\n```\n\n## Deterministic reports and agentic inference\n\n`catnip report` writes deterministic markdown from the store: totals and change, movers with trend, attribution, account events, audience, clone intent, coupling, depth, and data limitations.\n\nSame store and timeframe produce byte-identical output, and every figure is\ntagged `measured`. A second report over an unchanged store refuses to write —\ncatnip collects daily, so that would be one finding printed twice. `--force`\noverrides it; `--stdout` prints without writing and is never gated.\n\nA report is named for the period it covers, and its name says whether the\nfigures can still change:\n\n```\nreports/2026-08-05-2w.20260807T031722Z/   a day in the window can still be revised\nreports/2026-08-05-2w/                    none of them can\n```\n\nGitHub keeps correcting a day for well over a day after it closes, so a\nreport gets recomputed under a new write stamp when the store moves under\nit. The earlier copies stay. Once the period leaves GitHub's 14-day window\nnothing can revise it again, and the newest recomputation is moved to the\nunstamped name. `catnip report --locate` prints the current paths.\n\n`catnip-prowl` is the inferential pass: hypotheses the report doesn't ask,\ntested against the raw data, refutations reported with the findings. It\nwrites `prowl.md` beside `report.md`, so the reproducible file diffs clean.\n\n| tag | means | the test |\n|---|---|---|\n| `measured` | straight from `catnip.derive`, reproducible | it appears in `catnip report` |\n| `inferred` | a hypothesis that survived checking | the test and its result are stated |\n| `speculative` | a pattern worth watching, unsupported | what would confirm or kill it is stated |\n\n## See it run\n\n*Visualizing data with `catnip tui` after simulated `catnip run` data collection.*\n\n<img src=\"docs/media/demo.gif\" alt=\"Animated GIF of a terminal running 'catnip tui' over a 43-repository account. The traffic view opens on daily views and clones as bar charts, 8.9k views in the last 14 days. The audience view classifies each repo as audience, mixed, crawler or low-signal from its clones-per-unique-visitor ratio, and pressing ? opens a derivation overlay giving the formula, weights, thresholds and withheld components behind that score. The attribution view lists 103 findings of what moved and the release or push that plausibly caused it, tiered direct, coupled, account, dip, unexplained and no-effect; space opens one in full - a no-effect finding where five commits shipped and nothing followed, with the daily series, the median, MAD, mean absolute deviation and z-score the tier rests on, and what else was true that day. The deltas view shows signed windowed change and per-day rate, and space opens momentum: the daily level, the day-over-day derivative around a zero line, and a fitted slope. The anomaly view is a repo-by-day heatmap above a sortable list of account events, and tab then space opens one - the 31st of July, nine repos departed from their own normal and two of them shipped something that day. The funnel view shades each repo by its own busiest content category with a depth column, and space opens that repo's actual pages with views and uniques. The demo closes on the repo table and a repo drilldown of a crawler: 318 clones in one day against a baseline near 15, 158 unique cloners against 5 unique visitors, and a CONFLICT flag where the clone-intent score reads developer while the audience classification reads crawler.\"/>\n\n## TUI\n\n### Primary\n\n| Key | View | Answers |\n|---|---|---|\n| `1` | traffic | daily clones and views, account-wide; at `epoch`, the whole store plus stars/month |\n| `2` | audience | is this repo's traffic people or fetchers? |\n| `3` | table | every repo, sortable by momentum, audience, depth, stars-per-visitor |\n| `4` | lang | bytes by language |\n| `5` | attribution | what moved, and the release or push that plausibly caused it |\n| `6` | deltas | what changed this window versus last, and the per-day rate |\n| `7` | anomaly | repo × day heatmap; simultaneous spikes fold into one account event |\n| `8` | profile | clone intent — how many of the people who looked, cloned |\n| `9` | correlation | repos coupled after the account-wide release wave is removed |\n| `0` | funnel | content mix per repo, and how far past the front door traffic got |\n\n`1`–`0` jump | `v`/`V` cycle | `t`/`T` window (1d / 1w / 2w / all /\nepoch) | `/` filter | `s` sort | `q` quit.\n\n### Detail\n\n| view | `[space]` opens |\n|---|---|\n| most repo rows | the repo **drilldown** — daily charts with anomaly markers and release rules, uniques track, funnel mix, coupled repos, activity |\n| `5` attribution | the **finding** — the statistics the tier rests on, the borrowed cause if any, what else was true that day |\n| `6` deltas | **momentum** — level, day-over-day derivative, fitted slope, rate vs previous window |\n| `7` anomaly (`tab`) | the **account event** — every repo that moved that day: Z, value, median, cause |\n| `9` correlation | the **pair** — both daily series, both residual series, what residualizing changed |\n| `0` funnel | that repo's **actual pages** — views, uniques, paths |\n\n`j`/`k` next item | `esc` back one layer | `[enter]` always the repo drilldown.\n`S` flips sort direction; `o` cycles scope (owned / all / forks); `f` filters;\n`z` un-collapses unchanged delta rows; `l` shows low-signal repos.\n\n`[?]` shows the derivation behind the current view: formula, thresholds,\ninputs, raw-vs-unique, withheld components.\n\n### Headless\n\n`catnip view <name>` reads the same functions the screens do;\n`catnip view why <view>` prints a derivation. The viewer runs with\nno run directory at all: `--store-only`, `--history-file`, `--stats-file`.\n\nThat is the state `catnip prune` eventually leaves behind, and it is the one\nconsequence `CATNIP_RETAIN_DAYS` has beyond disk. Every *windowed* number\nsurvives, because `derive.py` computes it from the store: traffic, audience,\ndeltas, anomalies, attribution, coupling, intent, momentum.\n\nOne view goes empty: `4:lang`. Language bytes are current state and come back\non the next run, so losing them costs nothing.\n\nPopular paths used to go with it, and that one was a real leak —\n`/traffic/popular/*` serves a rolling ~14 days, so a pruned run took the only\ncopy of observations GitHub will not serve again. Paths are ingested as of\nstore schema 3, dated the day they were observed, exactly as referrers are.\nThe funnel now reads the store and survives pruning.\n\n## Data, sources, storage, and retention\n\n| Data | Source |\n|---|---|\n| Stars, forks, watchers, languages, topics, license | `/repos/{owner}/{repo}` |\n| Clone counts + 14 daily buckets, unique cloners | `/traffic/clones` |\n| View counts + 14 daily buckets, unique visitors | `/traffic/views` |\n| Popular paths, referrers | `/traffic/popular/*` |\n| Star and fork events with timestamps | `/stargazers`, `/forks` |\n| Commit activity, code frequency, contributor stats | `/stats/*` |\n| Pull requests, issues, releases and asset downloads | `/pulls`, `/issues`, `/releases` |\n\n### Data retention\n\nA **run** is one fetch's raw API bodies and derived CSVs. \nRuns age out because once their days are in the store they are redundant. \nRun retention is configured with `CATNIP_RETAIN_DAYS`.\n\nThe **store** is the permanent daily series.\nStore path is configured with `CATNIP_DATA_DIR`. \n\n```\nruns/20260805T031722Z/\n  raw/          one JSON body per endpoint per repo, as returned\n  analysis/     22 CSVs + summary.md — the schema everything else reads\n  reports/      which repos were skipped, and which denied traffic\n  manifest.json owner, counts, rate-limit spend, duration\nstats/\n  totals.json           account-wide rollup, rebuilt from scratch each run\n  history/\n    traffic_daily.json  the permanent series — never pruned. Daily clones\n                        and views per repo (count and uniques), plus dated\n                        referrer observations and a release/push event log\n    snapshots.jsonl     append-only per-run, per-repo snapshots\n    daily_snapshots.jsonl  what each fetch read for each still-movable day,\n                        before the merge destroyed it. The one input\n                        `catnip settle` has; bounded by\n                        CATNIP_SETTLE_LOG_DAYS and safe to delete\n```\n\n## Configuration\n\n`catnip config` shows active configuration\n\n* Example: [`catnip.conf.example`](catnip.conf.example)\n* Documentation: [docs/configuration.md](docs/configuration.md) \n\n```ini\nCATNIP_OWNER=octocat\nCATNIP_EXCLUDE=dotfiles *-private\nCATNIP_INCLUDE_FORKS=false\nCATNIP_RETAIN_DAYS=30           \nCATNIP_TIMER_ONCALENDAR=daily\n```\n\n## Automation\n\n```bash\ncatnip timer install     # systemd --user timer, daily, Persistent=true\ncatnip timer status      # next run, last result\ncatnip timer logs        # journalctl for the service\n```\n\nUser timer, not system — `gh` reads credentials from your keyring, which root\ndoesn't have. The installer checks user lingering and prints the fix. No\nsystemd: `catnip timer cron`. macOS/launchd and the silent-failure modes:\n[docs/automation.md](docs/automation.md).\n\n### Tannery\n\nA timer collects. [`tannery/`](tannery/) is the same schedule with agents on\nit - a [leather](https://github.com/TGPSKI/leather) tannery.\n\n```bash\ncd tannery\nmake validate     # leather validate every agent, lifecycle and toolset\nmake smoke-tools  # exec the read-only tools for real\nmake serve        # run the scheduler\n```\n\n| agent | when | does |\n|---|---|---|\n| `catnip-collect` | daily 05:07 | `catnip run`, then checks it landed |\n| `catnip-report` | daily 06:52 | writes the deterministic report |\n| `catnip-prowl` | every 3rd day 08:22 | hunts what the report does not answer |\n\nEach agent is multi-turn and every turn replaces its tool scope, so an agent\ncan only reach the tools that turn declares:\n\n```\ncatnip-prowl   catnip-evidence -> catnip-file -> catnip-publish\n```\n\nIt cannot file a finding during the turn it gathers evidence, and cannot\ngather more once it starts filing. \"Test before you file\" is not an\ninstruction it is asked to follow - there is no turn in which it can do\notherwise. See [tannery/README.md](tannery/README.md).\n\n## Command reference\n\n| | |\n|---|---|\n| `catnip init` | Write a config file (interactive, or fully from flags) |\n| `catnip doctor` | Tooling, credentials, scopes, traffic access, data. `--json` for agents |\n| `catnip run` | The whole pipeline; what the timer runs |\n| `catnip fetch` | Collect a new run (`--dry-run` previews the repo selection) |\n| `catnip analyze` | Re-derive analysis CSVs for a run |\n| `catnip history` | Ingest runs into the permanent store |\n| `catnip totals` | Rebuild account-wide totals |\n| `catnip verify` | Assert the analysis CSVs the TUI requires exist and are non-empty |\n| `catnip tui` / `view` | Interactive UI / one view as text |\n| `catnip report` | Deterministic markdown analysis of the store (`--stdout`, `--force`) |\n| `catnip summary` | The newest run's `summary.md` |\n| `catnip settle` | Measure how long GitHub kept revising each recent day; exits non-zero when that contradicts `CATNIP_SETTLE_HOURS` |\n| `catnip prune` | Delete run directories past their retention — never one whose traffic days are missing from the store. Dry-run by default |\n| `catnip timer` | `install` · `status` · `logs` · `uninstall` · `print` · `cron` |\n\n## Go deeper\n\n| you want to… | start here | then |\n|---|---|---|\n| **collect** reliably, on a schedule | [docs/automation.md](docs/automation.md) | [docs/configuration.md](docs/configuration.md) · `catnip doctor` |\n| **understand** a number on the screen | `[?]` on that view | [docs/metrics.md](docs/metrics.md) · `src/catnip/derive.py` |\n| **read** the data without the TUI | `catnip report --stdout` | `catnip view <name>` · [`catnip-prowl`](.agents/skills/catnip-prowl/SKILL.md) |\n| **contribute** (human or agent) | [AGENTS.md](AGENTS.md) | [CONTRIBUTING.md](CONTRIBUTING.md) · [CHANGELOG.md](CHANGELOG.md) · [LINEAGE.md](LINEAGE.md) |\n\n## License\n\nGPL-3.0 — see [LICENSE](LICENSE)."}