{"id":"0e5c528c-4ef2-426b-949e-999ca85057bd","slug":"incident-agent","kind":"mcp","name":"incident-agent","summary":"Claude Code agent that investigates production incidents: subagents, a skill, a custom MCP server and hooks","owner":"Dominik2806","repoUrl":"https://github.com/Dominik2806/incident-agent","license":"MIT","licenseTier":"permissive","trust":"community","stars":0,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-10-15T10:44:04.000Z","addedAt":"2026-10-21T14:34:59.096Z","logoUrl":"https://avatars.githubusercontent.com/u/130377056?v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["developer-tools","databases","cloud-infra"],"tags":["ai-agents","claude-code","incident-response","python","hooks","sqlite"],"runtime":"python","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-26T11:56:22.533Z","sample":false,"url":"https://conduits.dev/tools/incident-agent","language":"Python","transports":["stdio"],"install":{"pypi":"incident-agent"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# incident-agent\n\n[![CI](https://github.com/Dominik2806/incident-agent/actions/workflows/ci.yml/badge.svg)](https://github.com/Dominik2806/incident-agent/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n[![Python 3.12](https://img.shields.io/badge/python-3.12-blue.svg)](pyproject.toml)\n\nA Claude Code agent that works like an on-call engineer: it investigates a production error from logs, finds the\nroot cause, proposes a fix and writes an incident report. The agent proposes, the human decides.\n\n## What it does\n\nYou report a problem, for example *\"Since about 2 pm customers are getting 500 errors at checkout. What is going\non?\"*. Claude follows the `incident-runbook` skill: three read-only subagents triage the incident, find the root\ncause and propose a patch plus a rollback, Claude assigns a severity (SEV-1 to SEV-4) and writes a report to\n`reports/`. Nothing is applied automatically.\n\nNo production system is needed. A script generates a day of deterministic sample logs with a built-in incident into\nSQLite, and a custom MCP server makes them queryable. Everything runs locally.\n\n## Requirements\n\n- [uv](https://docs.astral.sh/uv/) (it installs Python 3.12 and the dependencies on first use)\n- [Claude Code](https://claude.com/claude-code)\n\n## Quickstart\n\n```bash\ngit clone https://github.com/Dominik2806/incident-agent.git && cd incident-agent\nuv run generate-data\nclaude\n```\n\nThen paste the demo prompt:\n\n> Since about 2 pm customers are getting 500 errors at checkout. What is going on?\n\nOn first start Claude Code asks you to **trust the workspace** (required for the hooks to run) and to **approve the\nproject MCP server** `incident-logs` from [`.mcp.json`](.mcp.json) (one time). `/mcp` should then list it with four\ntools. The expected result: `payment` v1.4.2 (deployed 14:01) raises `KeyError: 'currency'` for legacy clients,\nseverity SEV-2, a patch and a rollback as proposals, and a report in `reports/`.\n\n## What is inside\n\n| Part | What it does | Where |\n|---|---|---|\n| Subagents | `triage` (what, since when, how bad), `root-cause` (cause with evidence), `fix-proposer` (patch as diff, rollback, risks; only `Read`, `Grep`, `Glob`) | [`.claude/agents/`](.claude/agents/), [docs](docs/subagents.md) |\n| Skill | `incident-runbook`: the procedure, the severity matrix and the report template | [`.claude/skills/incident-runbook/`](.claude/skills/incident-runbook/SKILL.md) |\n| MCP server | `incident-logs`, read-only: `get_error_stats`, `get_deployments`, `search_logs`, `get_stacktrace` (max. 200 rows per call) | [`src/incident_agent/`](src/incident_agent/), [docs](docs/mcp-tools.md) |\n| Safety hooks | `block_destructive.py` (blocks `rm -rf`, `DROP`, `git push --force`, ...), `protect_files.py` (`.env`, `*.db`, `data/`, CI workflows), `audit_log.py` (every tool call to `logs/audit.jsonl`) | [`.claude/hooks/`](.claude/hooks/), [docs](docs/hooks.md) |\n| Notification hook | `notify.py`: desktop notification and optional Slack message (`SLACK_WEBHOOK_URL`) | same |\n| Stop hook | `check_report.py`: keeps Claude working until the report has every section of the template (one correction round) | same |\n| Demo app | `demo_app/`: tiny service with the intentional bug, so the fix proposal points at real code | [docs](docs/demo-app.md) |\n\nHow the parts fit together: [docs/architecture.md](docs/architecture.md). Why they are built this way:\n[docs/adr/](docs/adr/README.md).\n\n## Scenarios\n\n| Scenario | Story | Demo prompt |\n|---|---|---|\n| `checkout-500` (default) | Bad `payment` deployment: `KeyError: 'currency'` breaks checkout for legacy clients. Expected: SEV-2 | Since about 2 pm customers are getting 500 errors at checkout. What is going on? |\n| `db-pool` | Traffic spike exhausts the `catalog` connection pool; the `auth` deployment is a red herring. Expected: SEV-3 | Since mid-afternoon (around 3 pm UTC) product searches have been slow or failing for some customers. What is going on? |\n| `cert-expired` | The TLS certificate between `checkout` and `payment` expires at 15:20 UTC and almost every checkout fails; the `auth` deployment is a red herring. Expected: SEV-1 | Since about 3:20 pm UTC almost every checkout attempt is failing for our customers. What is going on? |\n| `api-outage` | The external card processor `Payvia` is down from 16:05 to 17:20 UTC and about 4 in 10 checkouts fail; the `checkout` deployment is a red herring and the incident recovers on its own. Expected: SEV-2 | Earlier today, around 4 pm UTC, some customers could not pay at checkout. What happened? |\n| `memory-leak` | `catalog` v2.9.0 leaks memory and the platform kills the container four times, each restart drops requests for 2 minutes; an `auth` deployment and an older cache deployment are red herrings and the leak is still running. Expected: SEV-3 | Several times today, the first around 1:15 pm UTC, customers got errors on product pages for a minute or two. What is going on? |\n\nSwitch with `uv run generate-data --scenario db-pool` (each run overwrites `data/incident.db`; no restart needed;\n`uv run generate-data` returns to the default). Details, timelines and expected causes:\n[docs/scenarios.md](docs/scenarios.md).\n\nExample reports written by the agent: [checkout-500](docs/examples/incident-report.md) and\n[db-pool](docs/examples/incident-report-db-pool.md).\n\n## Safety\n\nThe agent only proposes fixes, and its subagents have no `Write`, `Edit` or `Bash` tool. The hooks add deterministic\nrules on top, but be clear about their limits: **they are a safety net, not a security boundary.**\n\n- Guards are pattern based. Obfuscated or generated commands get through, and `protect_files.py` does not see writes\n  made through the shell. Harmless commands that merely contain text like `rm -rf` (for example in a heredoc) are\n  blocked too.\n- The audit, notification and stop hooks fail open on purpose; the stop hook blocks only once per stop.\n- Hooks only run after you trust the workspace, and anyone who can edit `.claude/settings.json` can change them.\n\nDetails: [docs/hooks.md](docs/hooks.md).\n\n## Development\n\n```bash\nuv sync\nuv run pytest\nuv run ruff check\nuv run ruff format --check\n```\n\n`main` is protected: one branch per change (`feature/`, `fix/`, `docs/`, `chore/`), Conventional Commits, squash-merged\npull requests with a CHANGELOG entry. See [CONTRIBUTING.md](CONTRIBUTING.md). Further reading:\n[roadmap](docs/ROADMAP.md), [changelog](CHANGELOG.md).\n\n## Roadmap status\n\n| Phase | Result | Status |\n|---|---|---|\n| 0 | Project skeleton, CI, templates | Done |\n| 1 | Log generator and `checkout-500` scenario | Done |\n| 2 | MCP server `incident-logs` | Done |\n| 3 | Subagents `triage` and `root-cause` | Done |\n| 4 | `incident-runbook` skill | Done |\n| 5 | Safety hooks (v0.1.0) | Done |\n| 6 | Demo app and `fix-proposer` | Done |\n| 7 | Notification and stop hooks, `db-pool` scenario | Done |\n| 8 | Docs and release v1.0.0 | Done (v1.0.0) |\n\nOutlook (not started): bundle everything as a Claude Code plugin, connect real data sources such as Sentry, run on a\nschedule, more scenarios, automatic evaluation of the expected\ncause per scenario.\n\n## License\n\n[MIT](LICENSE)\n"}