{"id":"e9752948-dc34-49de-9875-8b5aa2372307","slug":"agentic-code-reasoning-skills","kind":"skill","name":"agentic-code-reasoning-skills","summary":"Agentic Code Reasoning Skills - Structured semi-formal reasoning templates for AI-assisted code analysis","owner":"KunihiroS","repoUrl":"https://github.com/KunihiroS/agentic-code-reasoning-skills","license":"MIT","licenseTier":"permissive","trust":"community","stars":0,"match":null,"fewStars":true,"format":"skill","createdAt":"2026-03-28T07:39:53.000Z","addedAt":"2026-10-21T12:48:59.150Z","logoUrl":"https://avatars.githubusercontent.com/u/16163323?u=02d8b5916fb57ff38ad5a3f6e185eaefda1b02b5&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["developer-tools"],"tags":["claude-code","code-analysis","code-reasoning","llm-agents","prompt-engineering","semi-formal-reasoning","swe-bench"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-23T19:40:05.106Z","sample":false,"url":"https://conduits.dev/tools/agentic-code-reasoning-skills","language":"Shell","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# Agentic Code Reasoning Skills\n\nAn agent skill that translates the **Agentic Code Reasoning** paper into a practical, unified workflow for evidence-based code analysis without executing repository code.\n\nFor design rationale and a deeper explanation of how the paper was translated into the skill, see [`docs/design.md`](docs/design.md).\n\n## Original Paper\n\n**Agentic Code Reasoning** by Shubham Ugare and Satish Chandra (Meta)\n- arXiv: https://arxiv.org/abs/2603.01896\n- Key results: semi-formal reasoning improves accuracy by 5–12 percentage points across patch equivalence verification, fault localization, and code question answering tasks\n\n## Installation\n\nCopy `SKILL.md` into your project's Claude Code skills directory:\n\n```bash\n# Create the skills directory if it doesn't exist\nmkdir -p .claude/skills\n\n# Option 1: Download directly from GitHub\ncurl -o .claude/skills/agentic-code-reasoning.md \\\n  https://raw.githubusercontent.com/KunihiroS/agentic-code-reasoning-skills/main/SKILL.md\n\n# Option 2: Clone and copy\ngit clone https://github.com/KunihiroS/agentic-code-reasoning-skills.git\ncp agentic-code-reasoning-skills/SKILL.md .claude/skills/agentic-code-reasoning.md\n```\n\nThe skill activates automatically when Claude Code detects code reasoning tasks (e.g., \"are these equivalent?\", \"where is the bug?\", \"what does this code do?\").\n\n## What This Is\n\nThis repository contains a single agent skill (`SKILL.md`) that implements **semi-formal reasoning** — a structured prompting methodology where the agent must state explicit premises, trace concrete code paths with file:line evidence, and derive formal conclusions before making any claim.\n\nThe goal is to make code reasoning more reliable by forcing evidence gathering before judgment. The skill is intended for static code analysis tasks where careful tracing matters more than quick intuition.\n\nThe skill supports four modes:\n\n| Mode | Purpose |\n|------|---------|\n| `compare` | Determine if two code changes produce the same behavioral outcome |\n| `diagnose` | Find the root cause of a single defect from a failing test or error report |\n| `explain` | Answer a code question with verified semantic evidence |\n| `audit-improve` | Review code for security vulnerabilities, API misuse, or maintainability issues |\n\n## How It Works\n\nEach mode follows a shared **certificate-based reasoning process**:\n\n1. **Numbered premises** — state known facts before exploring\n2. **Hypothesis-driven exploration** — form expectations before reading files, record observations after\n3. **Interprocedural tracing** — read actual function definitions, never infer from names\n4. **Mandatory refutation** — try to disprove the conclusion before finalizing\n5. **Formal conclusion** — reference specific premises and claims by number\n\nThe key idea from the paper: structured templates act as *certificates* that prevent the agent from skipping cases or making unsupported claims.\n\n## Changes from the Original Paper\n\n### `localize` → `diagnose` (renamed)\n\nThe original paper evaluated fault localization on **Defects4J** (single-defect Java bugs where the root cause resides in 1–5 files). The mode was named `localize` following the paper's terminology.\n\nHowever, in practice, \"localize\" is ambiguous — it can be interpreted as \"list all files related to a problem,\" which includes broad structural changes, large-scale refactoring, and multi-file feature additions. When this skill's structured analysis (premise → trace → refutation → conclusion) is applied to such broad enumeration tasks, it **over-constrains the output**: the certificate-based process forces the agent to produce only high-confidence, evidence-backed predictions, which reduces file coverage compared to unrestricted exploration.\n\nThis was confirmed empirically: on SWE-bench Pro localization tasks with 17–106 ground-truth files, the skill reduced accuracy from 100% to 80% (with-skill vs without-skill), because the structured analysis narrowed predictions to fewer files and missed the breadth required.\n\n**The rename to `diagnose` makes the scope explicit:** this mode is for diagnosing a single defect whose root cause is in a small number of files, not for broad file enumeration.\n\n### Activation gates (added)\n\nTo prevent misapplication, the skill now includes **activation gates** — conditions that must be met before the skill is invoked. If the task requires broad file enumeration, large-scale refactoring, or directory-level reorganization, the skill explicitly recommends not activating. This protects against cases where structured analysis degrades performance compared to unrestricted exploration.\n\n## Reliability Evaluation\n\nEvaluated on two benchmarks using `claude-haiku-4.5` as the benchmark model (with-skill vs without-skill ablation):\n\n### Compare Pro (SWE-bench Pro, 20 pairs, multi-language)\n\n20 patch pairs from Go/JS/TS/Python repositories (10 EQUIVALENT + 10 NOT_EQUIVALENT).\n\n| | without skill | with skill | Delta |\n|---|---|---|---|\n| **Overall** (avg of 6 runs) | 59.2% | **64.2%** | **+5.0pp** |\n| stdev | 8.6% | 10.5% | — |\n\n### Audit-Improve (SWE-bench Pro security bugs, 28 tasks)\n\n28 security vulnerability localization tasks across ansible, flipt-io, teleport, vuls, NodeBB, etc. (Go, Python, JS, TS).\n\n| | without skill | with skill | Delta |\n|---|---|---|---|\n| **File+func match** (avg of 4 runs) | 82.1% | **88.4%** | **+6.3pp** |\n| stdev | 0% | 3.5% | — |\n\n### Key findings\n- The skill provides consistent positive effect across both benchmarks.\n- Audit-improve shows stable improvement with near-zero variance on the without-skill baseline.\n- Compare Pro has higher variance; the skill also reduces output variance (stdev 10.5% vs 8.6% without).\n- 118 iterations of automated self-improvement (Phase 1-2) converged statistically; structural changes to the SKILL.md (STRUCTURAL TRIAGE, Activation gates) provided measurable improvements.\n\nBenchmark scripts and compact input data are included under [`benchmark/swebench/`](benchmark/swebench/). Full raw outputs from automated self-improvement runs remain recoverable from the archived experiment branches and tags listed below.\n\n## Archived Experiment Branches\n\n`main` contains the current stable release of the skill. The historical auto-improvement branches are retained as remote branches for transparency, and their current heads have also been preserved with archive tags so their experimental context remains recoverable without treating them as release branches.\n\n| Archive tag | Branch preserved | Meaning |\n|---|---|---|\n| `archive/script-auto-improve-final` | `script/auto-improve` | Early automation/script development branch for the benchmark and patch-equivalence workflow. |\n| `archive/meta-agent-1-final` | `meta-agent/auto-improve` | First long-running meta-agent self-improvement branch; useful as historical evidence, not the release source. |\n| `archive/meta-agent-2-final` | `meta-agent-2/auto-improve` | Phase 1-2 compare-mode convergence branch. Its iter-46 `SKILL.md` is useful evidence for compare-mode improvements, but it is not the canonical full multi-mode skill by itself. |\n| `archive/meta-agent-3-final` | `meta-agent-3/auto-improve` | Later GPT-5.5/meta-agent-3 compare-mode exploration branch; retained for analysis, but its HEAD did not supersede the iter-46 compare evidence. |\n\nThe marker `v1.1.0-iter46` points to the iter-46 compare-mode candidate commit for historical/reproducibility purposes. `main` preserves the full multi-mode skill (`compare`, `diagnose`, `explain`, and `audit-improve`) rather than publishing a compare-only auto-improvement artifact. Future experiments should use short-lived `experiment/*` branches and should only be promoted to `main` after checking that non-targeted modes have not been dropped.\n\n## Repository Structure\n\n```\n├── SKILL.md                    # The skill (install this)\n├── auto-improve.sh             # Automated self-improvement loop (Phase 1-3)\n├── prompts/                    # Externalized prompt templates (Phase 3)\n│   ├── manifest.json           # Template registry (vars, roles)\n│   ├── propose-normal.txt      # Proposal prompt (normal mode)\n│   ├── propose-escape.txt      # Proposal prompt (structural reform mode)\n│   ├── discuss.txt             # Discussion/review prompt\n│   ├── implement.txt           # Implementation prompt\n│   ├── audit.txt               # Audit prompt\n│   ├── revise.txt              # Revision prompt\n│   ├── update-bl.txt           # Failed-approaches update prompt\n│   └── meta-propose.txt        # Meta-agent prompt (edits other templates)\n├── benchmark/\n│   └── swebench/               # SWE-bench benchmark suite\n│       ├── data/               # Benchmark inputs (pairs, tasks)\n│       ├── runs/               # Results and archive.jsonl\n│       ├── select_parent.py    # score_prop parent selection (HyperAgents)\n│       ├── detect_stagnation.py # Stagnation detection for meta-agent trigger\n│       └── append_archive_entry.py # Archive writer with template versioning\n├── docs/\n│   ├── design.md               # Design rationale\n│   ├── iteration-workflow.md   # How to iterate on SKILL.md from main\n│   ├── reference/              # Original paper PDF\n│   └── archive/                # Historical plans (meta-agent design, branch consolidation, auto-improve update log)\n├── Objective.md                # Experiment objectives and audit rubric\n├── failed-approaches.md        # Accumulated failure principles\n└── LICENSE\n```\n\n## License\n\nSee [LICENSE](LICENSE).\n"}