{"id":"b83afa9e-7464-4406-90cf-24833af3c93e","slug":"scientific-knowledge","kind":"skill","name":"scientific-knowledge","summary":"Agent skill to download, organize, process and summarizes papers locally for research","owner":"ztjona","repoUrl":"https://github.com/ztjona/scientific-knowledge","license":"MIT","licenseTier":"permissive","trust":"community","stars":3,"match":null,"fewStars":true,"format":"skill","createdAt":"2026-08-06T19:20:23.000Z","addedAt":"2026-10-21T23:37:05.672Z","logoUrl":"https://avatars.githubusercontent.com/u/23049553?u=6e9f2c665c6e7c6c9a0ed549a69c5a89f95c7e40&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["memory-knowledge"],"tags":["ai-agents","arxiv","research"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-26T03:52:16.367Z","sample":false,"url":"https://conduits.dev/tools/scientific-knowledge","language":"Python","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# Scientific Knowledge Skill\n\nA [Claude Code / Copilot Agent](https://docs.claude.com/en/docs/claude-code) skill that\nturns your assistant into a research-paper librarian. Give it an ArXiv URL, an ID, or a\nlocal PDF, and it downloads, converts, summarizes, and files the paper into a personal\nknowledge database organized by topic — with BibTeX references, citation graphs, and\ncritical, honest summaries.\n\nDeveloped by **Jonathan Zea**.\n\n## What it does\n\n- **Download & process ArXiv papers** — fetch metadata, download the PDF, convert it to\n  Markdown, append a BibTeX entry, and write a structured, critical summary.\n- **Ingest local (non-ArXiv) PDFs** — journal or conference papers you already have on\n  disk, using a BibTeX entry for metadata.\n- **Organize by topic** — papers are filed into topic folders you control (or the skill\n  infers from ArXiv categories), so the database stays navigable as it grows.\n- **Track a searchable database** — `abstracts.json` indexes every paper; a local regex\n  `search` command finds papers by tag, title, author, topic, or abstract without ever\n  hand-parsing JSON.\n- **Build a citation graph** — automatically extracts which stored papers cite each\n  other, so you can see how ideas in your database connect.\n- **Discover new papers** — search ArXiv by keyword, automatically excluding anything\n  already in your database.\n- **Synthesize across papers** — generate topic-level or cross-topic synthesis\n  documents that surface shared assumptions, contradictions, and open gaps.\n- **Preserve your notes** — every summary has a `## My Notes` section that survives\n  regeneration, so your personal annotations are never overwritten.\n- **Grade the evidence, not just the claims** — every summary carries an `Evidence basis`\n  and a `Headline vs evidence` verdict in its header, so an overclaimed paper is visible\n  at a glance instead of only in the fine print.\n- **Batch process** — a dedicated agent mode processes long paper lists unattended,\n  without burning your main chat's context window.\n\n## Epistemic header fields\n\nSummaries are read in bulk — a topic synthesis may sweep 150 of them — so the header\ncarries the information a skimming reader must not miss. Every summary written under\ntemplate version 3 or later includes:\n\n| Field | Values | Purpose |\n|---|---|---|\n| `Evidence basis` | `proof`, `intervention`, `correlational`, `benchmark`, `artifact`, `case-study`, `survey` — list-valued, primary first | What actually licenses the paper's central claim. Field-independent: `intervention` covers activation patching, a gene knockout, and a randomized trial alike — the authors moved something and measured the effect. `correlational` means they only looked. |\n| `Headline vs evidence` | `overclaimed`, `calibrated`, `understated` | Whether the title matches what was shown. A paper asserting a mechanism on purely associational evidence is `overclaimed`, however good the work. |\n| `Template version` | integer, currently `3` | Which summary template produced the file, so you can find and upgrade older ones. Summaries without the field predate versioning. |\n\nThe point is comparability. Two papers can both claim an internal representation exists —\none having ablated it, the other having merely decoded it — and without these fields both\nread as equally settled once compressed into a synthesis. Grepping `correlational`\ntogether with `overclaimed` returns precisely the citations to handle with care.\n\nSummaries also open with a `## Bottom Line` block restating the central claim at the level\nthe evidence supports, plus what went untested, before any of the detail.\n\n## Requirements\n\n- Python 3.10+\n- Packages: `docopt`, `pyyaml`, `requests`, `markitdown`, `pypdf`\n\n```bash\npip install -r requirements.txt\n```\n\n- Internet access (for the ArXiv API and PDF downloads)\n\n## Installation\n\nClone or copy this folder into your skills directory, for example:\n\n```\n~/.claude/skills/scientific-knowledge/\n```\n\nThen set the environment variable your `config.yaml` points at (default: `PAPERS_DB`)\nto wherever you want your paper database to live on this machine:\n\n```bash\nexport PAPERS_DB=\"$HOME/papers/agent\"      # macOS/Linux\nsetx PAPERS_DB \"%USERPROFILE%\\papers\\agent\" # Windows\n```\n\n`papers_folder` can itself be its own git repository if you want the database\nversion-controlled and synced separately from the skill.\n\n## Configuration\n\nAll settings live in [`config.yaml`](config.yaml):\n\n| Key | Description |\n|---|---|\n| `papers_folder` | Where papers, PDFs, markdown, and the database are stored. Use an environment variable reference — never a machine-specific absolute path — since this file is meant to be shared/tracked in git. |\n| `arxiv_api_url` | ArXiv API endpoint (rarely needs changing). |\n| `default_topic` | Fallback topic folder when none is specified. |\n| `pdf_engine` | `markitdown` (default) or `pypdf` (fallback). |\n\n## Usage\n\nOnce installed, just talk to your assistant naturally:\n\n- *\"Summarize this paper: https://arxiv.org/abs/2301.12345\"*\n- *\"Add this PDF to my database, here's the BibTeX...\"*\n- *\"Find papers in my database about sparse autoencoders\"*\n- *\"Discover recent papers on mechanistic interpretability\"*\n- *\"Give me a synthesis of everything I have on reinforcement learning\"*\n\nThe assistant follows the workflows and rules defined in [`SKILL.md`](SKILL.md), which is\nloaded automatically when the skill is relevant.\n\nFor batch processing many papers at once, use the dedicated **Scientific Knowledge**\nagent mode instead of the main chat — it runs in its own isolated context window.\n\n## Project structure\n\n```\nconfig.yaml              # Machine-specific configuration (env-var based)\nSKILL.md                 # Instructions the assistant follows (workflows, rules, template)\nrequirements.txt          # Python dependencies\nreferences/\n  REFERENCE.md            # Full folder layout, database schema, and CLI reference\nscripts/\n  arxiv_download.py       # fetch / process / discover / search / citations (ArXiv)\n  process_local.py        # process (local, non-ArXiv PDFs via BibTeX)\n```\n\nSee [references/REFERENCE.md](references/REFERENCE.md) for the complete database schema\nand CLI reference for both scripts.\n\n## License\n\n[MIT](LICENSE)\n\n## Author\n\n**Jonathan Zea**\n"}