knowledge-oss

knowledge-oss

A flat Obsidian vault that doubles as persistent LLM memory: ingest agent sessions and research exports, auto-index, and query from any project.

What this is

A starter kit for keeping a personal knowledge vault that an LLM agent can read, write, and search. The vault is plain markdown in a flat directory, indexed by filename prefix and Obsidian wikilinks. One Python package — python3 -m vault — handles indexing, linting, ingest, and a retrieval eval harness. A memory/ directory gives the agent durable identity, preferences, and project state across sessions. CLAUDE.md / AGENTS.md tells the agent how to use it all.

No embeddings, no database, no service. Markdown and git.

What it gives you

• One CLI — vault index, vault lint, vault ingest <source>, vault recurrence, vault eval, all stdlib-only (pytest is the sole dev dependency)
• Config, not forks — .vault.toml marks the vault root and declares everything else: sources, vaults, MOC budget, glossary prefixes, project buckets
• Source-agnostic ingest — a source is a directory of markdown you declare; manifests track what's been processed by SHA-256, so re-runs are idempotent and edited files resurface. Optional adapters bridge tools that store history as something else
• Multi-vault routing — declare two or more vaults with keywords and every ingested note lands in the right one, with an optional model tiebreak
• Token-cheap retrieval — a generated llms.txt router + backlinks.md let an agent find the right notes by reading ~1k tokens instead of the whole vault
• A lint gate — links, frontmatter schema, grounding, and file anchors, exiting non-zero so CI can hold the line
• Memory layout — memory/{user,preferences,decisions,people,current,future}.md
  • memory/daily/
• Append-only log — log.md is the changelog of vault operations
• Agent skills — .agents/skills/ (harness-neutral), symlinked into .claude/skills/ for Claude Code

Quickstart

# 1. Clone where you want your vault to live
git clone https://github.com/antigluten/knowledge-oss.git ~/knowledge
cd ~/knowledge

# 2. Make it yours: edit .vault.toml (name, raw-sources path, project buckets)

# 3. Open it in Obsidian (optional but recommended)
#    File → Open vault → ~/knowledge

# 4. Start an agent session in the vault
claude          # or codex, or pi

# 5. Tell it about you: edit memory/user.md, memory/preferences.md
#    (or just chat — the agent will offer to update these as it learns)

# 6. Generate the index
python3 -m vault index

To start ingesting, declare a source in .vault.toml — a directory of markdown under raw_sources:

[[source]]
name = "notes"

mkdir -p ~/raw-sources/notes     # drop .md files in here

python3 -m vault ingest notes list
python3 -m vault ingest notes mark <file> --slugs concept--foo,tool--bar

That's the triage flow: you see what hasn't been processed, write whatever notes it deserves, and record which slugs came from which file. For sources where one document is one note, mode = "auto" writes them for you:

[[source]]
name = "research"
mode = "auto"

python3 -m vault ingest research ingest

If your tool keeps history as JSONL or an export bundle rather than markdown, name a bundled adapter and the source gains an export step:

[[source]]
name = "sessions"
exporter = "codex"        # or "pi", "claude-chat"

python3 -m vault ingest sessions export   # native format -> markdown
python3 -m vault ingest sessions list     # then triage as usual

To check the vault stays healthy:

python3 -m vault lint          # links, schema, grounding, anchors
python3 -m vault index --check # fails if the committed index is stale
python3 -m pytest -q           # 234 tests, no network

Pointing another project at the vault

The vault is only useful if agents working elsewhere reach for it. Add a block like this to your harness's global instructions (~/.claude/CLAUDE.md, ~/.codex/AGENTS.md, ~/.pi/agent/AGENTS.md):

## Persistent knowledge vault

I keep a knowledge vault at `~/knowledge/`. Before answering non-trivial
questions about systems, tooling, or prior decisions, check it:

1. Read `~/knowledge/llms.txt` — the token-cheap router. Follow the retrieval
   protocol printed in it verbatim; it is the authoritative copy.
2. Open the one relevant MOC it names (the specific shard if the parent lists
   shards), then ≤3 notes chosen by their `description`.
3. Never read `glossary.md` or `backlinks.md` whole. Grep them:
   `rg -i "^\*\*<term>" ~/knowledge/glossary.md`, or
   `rg -A1 "## \[\[<stem>" ~/knowledge/backlinks.md` for what references X.
4. A note marked `[DEPRECATED]` is superseded — read its `superseded_by` target.
   A note marked `[STALE]` needs verifying before you rely on it.
5. Fall back to web search / fresh reasoning only if the vault doesn't cover it.

Don't write notes from outside the vault. To file something, start a session in
`~/knowledge/` and use its `ingest-to-vault` skill — a note written without that
contract fails `vault lint`.

How it works

The daily loop is:

sources → ingest → vault → query → synthesis → vault → lint → (repeat)

• Ingest: raw markdown (Perplexity exports, agent conversation logs) becomes one or more cleaned notes in the vault. Manifests track what has been processed, by SHA-256, so re-runs are idempotent and edited sources are re-flagged.
• Index: vault index walks the vault and regenerates per-domain MOCs (Maps of Content, auto-sharded when over the token budget), the alphabetical glossary.md, an llms.txt router (a ~1k-token domain map + top hubs an agent reads first), and backlinks.md (the inverted wikilink graph for entity lookup) — all from frontmatter + the > **TL;DR:** … line in each note.
• Query: the agent reads the vault from inside the directory — index files, grep, and full-text search are all available.
• Synthesize: when a session produces a useful cross-note answer, it gets filed as synthesis--<slug>.md so future sessions can rediscover it.
• Lint: vault lint sweeps for broken links, schema violations, ungrounded claims, and stale file anchors, and exits non-zero on any error.
• Recurrence: vault recurrence counts, per note, the later notes that link back to it — lessons the vault already held that got re-derived anyway. A prescriptive note with a growing count belongs in always-on config, not in a file someone has to think to open.

Vault prefixes

Notes are flat files; their filename prefix declares the kind of note:

The glossary column is the glossary_prefixes list in .vault.toml.

Configuring

.vault.toml is the single customization surface:

Its presence also marks the vault root — every vault command walks up from the current directory to find it, so the CLI works from any subdirectory and never guesses which vault you meant.

For the rest — prefix taxonomy, memory layout, daily-log format — see docs/customizing.md.

Optional: more than one vault

Omit [[vault]] and there is one vault: this directory. Declare two or more and every ingested note is routed to the one whose keywords it matches best — work in one, personal in another, reading notes in a third — with an optional model tiebreak for notes keywords can't place. See docs/multi-vault.md.

Setup guides

• docs/setup-llm-wiki-en.md — long-form English setup walkthrough
• docs/setup-llm-wiki-ru.md — Russian-language version
• docs/migrating-external-vault.md — how to fold an existing vault or journal into this layout

Credits

Based on the author's working knowledge vault. Public release as a starter kit so others can run the same pattern.

License

MIT — see LICENSE.