{"id":"4a093f60-f101-4f6b-bca3-d49ddbdb0831","slug":"sourcebraid","kind":"mcp","name":"sourcebraid","summary":"Weave the web into durable Markdown in your own GitHub repository.","owner":"patrickschiller","repoUrl":"https://github.com/patrickschiller/sourcebraid","license":"MIT","licenseTier":"permissive","trust":"community","stars":2,"match":null,"fewStars":true,"format":"mcp","createdAt":"2026-07-31T12:29:02.000Z","addedAt":"2026-10-21T04:54:58.592Z","logoUrl":"https://avatars.githubusercontent.com/u/39913537?u=4cedc6db73799cdecd08e675bd2b8c5f8fbffa79&v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["files-documents","browser-web","developer-tools"],"tags":["browser-extension","chrome-extension","codex-plugin","full-text-search","github","ios","knowledge-base","markdown"],"runtime":"binary","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-21T04:55:12.708Z","sample":false,"url":"https://conduits.dev/tools/sourcebraid","language":"Swift","transports":["stdio"],"install":{},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# SourceBraid\n\n**SourceBraid — Weave the web into Markdown.**\n\n[Deutsche Dokumentation](README.de.md)\n\n![SourceBraid — Weave the web into Markdown.](assets/branding/sourcebraid-social-card.png)\n\n[SourceBraid](https://sourcebraid.com) saves articles, research papers, wiki\npages, GitHub Gists, and PDF documents as durable Markdown in a private GitHub\nrepository. Metadata lives in YAML frontmatter, relevant images become local\nrepository assets, and every source is added to a searchable index.\n\nSourceBraid does more than bookmark URLs. It prepares each source using the\nrichest trustworthy representation available, keeps its provenance, and leaves\nyou with ordinary files and Git history that remain useful without SourceBraid.\n\n## How SourceBraid works\n\nSourceBraid combines capture clients, a private GitHub repository as the durable\nsource of truth, and two archive-access paths. The local Codex plugin supports\nsearch and archive management. The hosted, read-only ChatGPT integration is\nprepared for deployment and review; it is not yet a live public service. The\nChrome extension and iOS app write captures directly to the configured GitHub\nrepository. Hosted ChatGPT requests will pass through SourceBraid's Cloudflare\nWorker without creating a permanent content database.\n\nThe local SQLite index is only a rebuildable search cache. Markdown files and\nGit history remain authoritative.\n\n```mermaid\nflowchart TD\n    A[\"Web page, wiki, Gist, arXiv paper, or PDF\"] --> B{\"Capture client\"}\n    B -->|Chrome| C[\"Browser extension\"]\n    B -->|iOS| D[\"App and Share Extension\"]\n    C --> E[\"Select the best extraction adapter\"]\n    D --> E\n    E --> F[\"Normalize content, create frontmatter, and save images\"]\n    F --> G{\"PDF conversion required?\"}\n    G -->|No| H[\"Store Markdown, assets, and URL-hash metadata shard\"]\n    G -->|Yes| I[\"Store PDF, placeholder, and metadata\"]\n    I --> J[\"GitHub Action converts the PDF with Docling\"]\n    J --> H\n    H --> K[\"Private GitHub repository as source of truth\"]\n    K --> L{\"Local search index available?\"}\n    L -->|No| M[\"One-time index build\"]\n    L -->|Yes| N[\"Compare remote head and Git blob SHAs\"]\n    N -->|Changed| O[\"Download only new or changed files\"]\n    N -->|Unchanged| P[\"Use the existing index\"]\n    M --> Q[\"SQLite index with FTS5\"]\n    O --> Q\n    P --> Q\n    Q --> R[\"Local Codex: search, fetch, list, or safely delete\"]\n    K --> S[\"Prepared hosted MCP: read-only GitHub access\"]\n    S --> T[\"ChatGPT after deployment and review\"]\n```\n\nThe workflow in detail:\n\n1. **Capture:** Open SourceBraid on the current page or share content from iOS.\n   Add tags and personal notes before saving.\n2. **Extract:** SourceBraid selects the strongest available adapter. Structured\n   sources such as arXiv, Azure DevOps, Gists, and native Markdown take priority\n   over generic DOM extraction.\n3. **Prepare:** Content becomes portable Markdown. SourceBraid adds YAML\n   frontmatter, resolves relative links, and stores relevant images next to the\n   document so the clip remains readable without the original page.\n4. **Commit:** Documents, assets, and metadata are written through the GitHub\n   Contents API. Metadata is partitioned into up to 256 JSONL shards by URL hash.\n   Normal Git commits make every change inspectable and recoverable.\n5. **Finish PDFs:** When no suitable HTML representation exists, the original\n   PDF remains in the repository. A GitHub Action uses Docling to create the\n   final Markdown, extract figures, and replace the pending placeholder.\n6. **Index locally:** On first use, the Codex plugin builds a local SQLite FTS5 index. Later\n   updates compare the stored commit and Git blob SHAs, processing only new,\n   changed, or deleted files.\n7. **Use:** Local Codex searches the index and supports guarded deletion with a\n   preview and exact confirmation. Its last synchronized index remains readable\n   when GitHub is unavailable. The prepared ChatGPT service searches and fetches\n   through GitHub with read-only access, explicit coverage limits, and no local\n   index or deletion tools.\n\nKeeping the GitHub archive separate from the local search cache matters for\nlarge collections: a normal query does not need to reopen thousands of Markdown\nfiles. A full pass is needed only for the first build, an explicit rebuild, or\nindex repair.\n\n## Supported sources and conversion\n\n| Source or format | Preferred extraction | Markdown result | Images and attachments | Fallback |\n| --- | --- | --- | --- | --- |\n| **arXiv paper** | Experimental full-paper arXiv HTML | Sections, prose, tables, citations, and LaTeX formulas; authors, arXiv ID/version, DOI, categories, and journal reference in frontmatter | Figures are copied into the asset folder and linked relatively | Download the PDF and convert it with Docling |\n| **Remote or local PDF** | Original PDF plus asynchronous Docling workflow in GitHub Actions | Reading order, tables, OCR text, and referenced figures; starts as `pending`, then becomes finished Markdown | The original remains as `source.pdf`; extracted figures sit beside it | Local PDFs require Chrome's **Allow access to file URLs** setting; encrypted or session-only PDFs are unsupported |\n| **Azure DevOps Wiki** | Authenticated Wiki REST API returns source Markdown | Azure macros are normalized, Mermaid remains a `mermaid` code block, internal wiki links become absolute | Protected attachments are loaded through the still-authenticated source tab | Rendered `.markdown-content` area |\n| **GitHub Gist** | GitHub Gist API, with the configured token for private Gists | A single Markdown file directly; multiple files as sections; source code in language-tagged fences | Public images directly, protected GitHub images through the signed-in Gist tab | Revision-specific URLs keep their revision |\n| **Native Markdown** | HTTP response to `Accept: text/markdown`, for example from Hashnode or appropriately configured Cloudflare sites | Source frontmatter and duplicate H1 removed; relative links made absolute | Relevant images are stored locally and linked relatively | Continue through dedicated APIs, then DOM extraction |\n| **WordPress** | WordPress REST endpoint discovered from page metadata | Article content converted from structured API data | Relevant article images stored locally | Visible page content |\n| **Forem / DEV** | Forem API with source Markdown | Normalized Markdown without site chrome | Relevant images stored locally | Visible page content |\n| **Ghost** | Configured Ghost Content API | Structured post content with canonical URL validation | Relevant images stored locally | Visible page content |\n| **Blogger** | Blogger API using detected blog and post IDs | Structured article content | Relevant images stored locally | Visible page content |\n| **Google DeepMind blog** | Article sections from the page DOM | Full post without the cover controls or related-post cards | Article images stored locally | Generic visible-page extraction |\n| **JSON Feed, RSS, or Atom** | Feed announced by the HTML page | Full feed content when available | Relevant images stored locally | Visible page content |\n| **Generic HTML page** | Visible DOM, preferring `article`, `main`, or `[role=\"main\"]` | Headings, paragraphs, links, lists, quotes, code, and tables | Content-relevant images stored locally | `body` as the final fallback |\n\n### Detection order\n\nSourceBraid always uses the strongest available content source. For HTML pages,\nadapters run in this order:\n\n1. arXiv HTML\n2. Azure DevOps Wiki\n3. GitHub Gist\n4. Native Markdown\n5. WordPress REST\n6. Forem / DEV API\n7. Ghost Content API\n8. Blogger API\n9. Google DeepMind blog DOM\n10. JSON Feed, RSS, or Atom\n11. Visible DOM\n\nThe first matching, validated source wins. SourceBraid then normalizes the\nMarkdown, downloads images, writes YAML frontmatter, and updates the index.\n\n## Archive layout\n\nMarkdown files are stored through the GitHub Contents API:\n\n```text\nweb-clips/YYYY/MM/YYYY-MM-DD-domain-title-urlhash.md\n```\n\nRelated assets live under:\n\n```text\nweb-clips/YYYY/MM/assets/YYYY-MM-DD-domain-title-urlhash/\n```\n\nMarkdown image references are relative to this asset directory. PDF sources\nalso retain the original as `source.pdf`.\n\nSourceBraid maintains a URL-hash-sharded metadata index:\n\n```text\nweb-clips/index/00.jsonl\n...\nweb-clips/index/ff.jsonl\n```\n\nThe same URL always maps to the same shard, avoiding a rewrite of the entire\nmetadata collection on each capture. Existing archives with\n`web-clips/index.jsonl` remain compatible and can be migrated atomically through\nthe plugin. Each entry includes title, canonical URL, repository path, capture\ndate, optional publication and modification dates, tags, source type,\nextraction method, capture timestamp, and saved image paths.\n`date` and the `YYYY/MM` path use the local capture date; a source's publication\ndate remains separate in `published`.\n\n## Research papers and PDFs\n\n### arXiv directly to Markdown\n\nAn arXiv abstract page such as `https://arxiv.org/abs/2311.02462` can be saved\ndirectly. SourceBraid prefers the experimental HTML version of the full paper,\nconverts it to Markdown, and keeps research metadata. The PDF does not need to\nbe downloaded or opened manually.\n\nIf no HTML version exists, the extension uploads the PDF in the background and\nthe Docling workflow converts it automatically.\n\n### General PDFs\n\nSourceBraid accepts remote HTTP(S) PDFs and local `.pdf` files opened in Chrome.\nFor local files, enable **Allow access to file URLs** in SourceBraid's extension\ndetails at `chrome://extensions`. Without that permission, the extension shows\na concrete instruction instead of producing an empty HTML clip.\n\nA PDF capture initially stores:\n\n```text\nweb-clips/YYYY/MM/assets/CLIP-SLUG/source.pdf\n```\n\nThe extension creates a pending Markdown entry and metadata record. The final\nPDF commit starts `.github/workflows/convert-pdfs.yml`, which:\n\n1. installs Docling on a GitHub runner;\n2. extracts reading order, tables, OCR text, and figures;\n3. replaces the pending Markdown while preserving notes and frontmatter;\n4. marks the matching metadata entry as complete; and\n5. retains the original PDF beside extracted assets.\n\nGitHub Actions needs write access to repository contents. The workflow has a\n45-minute timeout, and individual PDFs are limited to 25 MB by browser and\nGitHub API constraints. Rerun a conversion under **Actions → Convert PDFs to\nMarkdown → Run workflow**.\n\nIf another clip is saved to the same branch during conversion, the workflow\nrefreshes its branch and retries a rejected push up to five times. Concurrent\nSourceBraid uploads are therefore not lost to a temporary Git ref race.\n\n## Wikis and Gists with images\n\n### Azure DevOps Wiki\n\nSourceBraid reads source Markdown through the authenticated Azure DevOps Wiki\nAPI. If that fails, it converts only the rendered `.markdown-content` area —\nnot navigation, headers, or unrelated Azure DevOps UI.\n\nProtected attachment URLs may need the browser's signed-in session, so\nSourceBraid loads images sequentially through the open source tab, stores them\nin the asset folder, and rewrites links to relative repository paths. Keep the\nsource tab open until capture completes. The frontmatter records the\norganization, project, wiki ID, page ID, page path, and revision when available.\n\n### GitHub Gists\n\nA one-file Markdown Gist becomes the document body directly. Multi-file Gists\nbecome one document with a section per filename; non-Markdown files remain in\nlanguage-tagged code fences.\n\nPublic Gists work anonymously. For private Gists, SourceBraid uses the configured\nGitHub token when it has Gist read permission. Signed-in GitHub image assets can\nbe loaded through the still-open Gist tab.\n\n## Chrome installation from source\n\nUntil the Chrome Web Store listing is live, load the reviewed extension from\nthis checkout:\n\n1. Open `chrome://extensions`.\n2. Enable **Developer mode**.\n3. Select **Load unpacked**.\n4. Choose [`chrome-extension/sourcebraid`](chrome-extension/sourcebraid).\n5. Open a supported source and select the **SourceBraid** icon.\n6. Configure the private GitHub repository, optionally add tags or notes, and\n   choose **Save to GitHub**.\n\nAfter setup, GitHub settings stay collapsed behind the settings icon. If only\nthe GitHub upload fails after successful extraction, the popup offers a\n**Download Fallback**. Before an upload starts, SourceBraid verifies that the\nconfigured repository exists and is accessible to the token; the popup reports\nan explicit error when GitHub returns `404 Not Found`.\n\nBefore the first capture, SourceBraid shows what page data is read and where it\ngoes, and requires affirmative consent. **Export Plugin Config** never includes\nthe GitHub token or optional source API credentials.\n\n## GitHub archive setup\n\nWith the GitHub CLI installed and authenticated (`gh auth login`), one command\ncreates or initializes the private archive for the authenticated account:\n\n```bash\npython3 scripts/setup_github.py\n```\n\nThe default target is `AUTHENTICATED_USER/sourcebraid-private`. The script\nrefuses public repositories, preserves existing files, enables GitHub Actions,\nuploads only the allowlisted PDF support files, and writes a token-free\n`sourcebraid-config.json`. Use `--repo OWNER/NAME` for another private target or\n`--dry-run` to preview the operation.\n\nRelease builds also provide the same workflow as one standalone file. Build it\nlocally with:\n\n```bash\npython3 scripts/build_setup_package.py\npython3 dist/sourcebraid-github-setup-v1.0.1.py --help\n```\n\n## GitHub token\n\nUse a fine-grained personal access token restricted to exactly one private\nrepository:\n\n```text\nContents: Read and write\nWorkflows: Read and write\n```\n\n`Workflows` is needed only for PDF support. On the first PDF upload, SourceBraid\ninstalls the bundled Docling workflow, conversion script, and requirements file\nwhen those paths do not already exist. Existing files are never overwritten.\nThe token is stored locally in Chrome extension storage.\n\nOptional API configuration:\n\n- Ghost Content API: base URL, such as `https://example.com/ghost/api/content`,\n  plus a browser-safe Content API key\n- Blogger: optional Google API key; public posts do not require OAuth, but\n  anonymous API calls normally need a key for quota\n\n## SourceBraid in ChatGPT and Codex\n\nThe primary ChatGPT release uses the hosted MCP service in [`chatgpt-mcp/`](chatgpt-mcp/).\nIts intended endpoint is `https://mcp.sourcebraid.com/mcp`; deployment, live\nauthentication tests, and OpenAI review/publication are still required. Follow\nthe [deployment guide](docs/CHATGPT_MCP_DEPLOYMENT.md) and\n[OpenAI MCP submission kit](marketing/OPENAI_MCP_SUBMISSION.md).\n\nAfter the service is deployed, users connect through SourceBraid's HTTPS consent\npage using a fine-grained GitHub token with **Contents: Read-only** for exactly\none private archive repository. Never paste that token into a ChatGPT prompt.\nThe hosted tools provide search, fetch, listing, and status; they cannot capture,\nedit, delete, migrate metadata, or build a local index. Default-branch search\nuses GitHub Code Search and verifies matches at a pinned commit. Indexing delays,\nlimits, and any bounded fallback are reported with the results.\n\nThe local Codex plugin remains the full archive-management option. It needs\nPython 3 and local GitHub authentication; a skills-only ZIP does not by itself\ngive ordinary ChatGPT access to those local files or credentials.\n\nFor local development, add this checkout as a marketplace and install the\nplugin:\n\n```bash\ncodex plugin marketplace add .\ncodex plugin add sourcebraid@sourcebraid\n```\n\nAfter the reviewed `v1.0.1` tag is published, the tag-bound GitHub installation\nis:\n\n```bash\ncodex plugin marketplace add patrickschiller/sourcebraid \\\n  --ref v1.0.1 \\\n  --sparse .agents/plugins \\\n  --sparse codex-plugin/sourcebraid\ncodex plugin add sourcebraid@sourcebraid\n```\n\nStart a new Codex conversation after installation.\n\n**Export Plugin Config** downloads `sourcebraid-config.json`. Store it at:\n\n```text\n~/.config/sourcebraid/config.json\n```\n\nOr configure the plugin in a terminal:\n\n```bash\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py config \\\n  --repo-slug OWNER/REPO --branch main --root-folder web-clips\n```\n\nThe versioned plugin lives under `codex-plugin/sourcebraid`. It uses a local\nSQLite FTS5 index, downloads only files whose Git blob SHAs changed, and\nsupports search, fetch, listing, and guarded deletion previews:\n\n```bash\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py index build\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py index update --max-age 900\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py index verify\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py search \"dynamic agents\" --tag ai\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py list \"dynamic agents\" --refresh\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py plan-delete \\\n  --path \"web-clips/2026/07/example.md\" --json\n```\n\nThe index is stored separately per repository, branch, and archive root under\n`~/.cache/sourcebraid/.../search.sqlite3` and is never committed. Search checks\nfor a changed remote head at most every 15 minutes; when GitHub is unavailable,\nthe local index remains usable. `search --scan` is an explicit `rg` diagnostic\nfallback.\nThe safer cache namespace introduced in this release triggers a fresh initial\nbuild; existing Markdown and Git history are unchanged.\n\nNew captures write stable URL-hash shards such as\n`web-clips/index/47.jsonl`. Legacy archives remain readable. Preview and confirm\nthe one-time migration against an unchanged branch head:\n\n```bash\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py index plan-shards --json\npython3 codex-plugin/sourcebraid/scripts/sourcebraid.py index migrate-shards \\\n  --expected-head HEAD_SHA --confirm-head HEAD_SHA --json\n```\n\nBefore deletion, the plugin shows the exact Markdown file, metadata change, and\nowned assets, then requires explicit confirmation. It writes a normal,\nnon-forced Git commit, so repository history remains recoverable.\n\nThe complete repository plugin includes the local stdio MCP server. The optional\nskills package contains the local Python workflows and is separate from the\nhosted **With MCP** submission. Requested sources are processed by the active\nChatGPT or Codex environment; hosted requests also pass through SourceBraid and\nCloudflare. See the [integration guide](docs/CHATGPT_PLUGIN.md) and\n[privacy notice](PRIVACY.md) for authentication, storage, and revocation details.\n\n## iOS\n\nThe native iOS app and Share Extension live under [`ios/`](ios/README.md).\nAfter one-time repository and token setup, URLs, selected text, Safari articles\nwith relevant image assets, PDFs, and other files can be sent to the configured\nprivate archive through the system share sheet.\n\nThe iOS **1.1.0** release candidate adds a Markdown reader, the latest seven\nclips, offline downloads and full-text search, optional on-device Apple\nIntelligence chat with source links, and adaptive iPad/iPhone Duo navigation.\nThe maintainer approved publication after a hands-on functional check on\nOctober 15, 2026. Xcode Cloud distribution and App Store availability are tracked\nseparately; approval does not establish a successful cloud build or store release.\nSee the [1.1.0 release notes](docs/IOS_RELEASE_1.1.0.md),\n[implementation and test record](docs/IOS_IMPLEMENTATION_TESTING.md), and\n[remaining device checks](docs/IOS_DEVICE_ACCEPTANCE.md). Native\nRecent/Search/Chat navigation and bounded background refresh are described in\nthe [UI and refresh concept](docs/IOS_READER_REFRESH_UX.md).\n\n## Android roadmap\n\nAndroid is intentionally not part of the first release. Feedback from the\nOpenAI community will determine whether an Android share target becomes the\nnext native client and which contributors or testers can help shape it.\n\n## Contributing and license\n\nSourceBraid is fully open source under the [MIT License](LICENSE). See\n[CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidance and DCO sign-off,\nthe [Code of Conduct](CODE_OF_CONDUCT.md) for community standards, and the\n[Security Policy](SECURITY.md) for private vulnerability reporting. The\n[public release process](RELEASING.md) documents versioning, validation, and\nreproducible release artifacts. The\n[privacy notice](PRIVACY.md), [terms](TERMS.md), and [notice](NOTICE) document\nthe capture, local-plugin, and hosted-service data flows and licensing boundaries.\n\n## Technical notes\n\nThe Chrome extension source lives under\n[`chrome-extension/sourcebraid`](chrome-extension/sourcebraid). It has no build\nstep and bundles no third-party runtime. Docling runs only inside the target\nrepository's GitHub Action. HTML conversion happens locally in the extension;\nAPI and image requests use either ordinary HTTP or the browser's existing\nauthenticated session, depending on the source.\n"}