{"id":"d0e597f2-7e37-4371-948b-cca8fb03e5b4","slug":"wet","kind":"mcp","name":"wet","summary":"Open-source MCP server for AI agents: web search, content extraction, and library docs -- 5-strategy scraping, runs without API keys.","owner":"n24q02m","repoUrl":"https://github.com/n24q02m/wet","license":"Apache-2.0","licenseTier":"permissive","trust":"community","stars":19,"match":null,"fewStars":false,"format":"mcp","createdAt":null,"addedAt":"2026-10-14T05:04:20.088Z","logoUrl":"https://avatars.githubusercontent.com/u/135627235?v=4","linkOnly":false,"collection":null,"skillPath":null,"categories":["cloud-infra","browser-web","search"],"tags":["crawl4ai","open-source","python","searxng","docker","web-scraping","web-search","ai-agents"],"runtime":"python","local":true,"cloudCapable":false,"cloudCapability":"unknown","cloudReadyOnPlatform":false,"needsApiKey":false,"reviewed":false,"syncedAt":"2026-10-14T05:08:33.913Z","sample":false,"url":"https://conduits.dev/tools/wet","language":"Python","transports":["stdio"],"install":{"pypi":"wet-mcp"},"requirements":{"env":[],"upstreamMayCharge":false},"tools":null,"readme":"# WET - Web Extended Toolkit MCP Server\n\n> **Renamed (2026-09-13):** repo is now `wet` — CLI-first (`wet` command). PyPI package stays `wet` (PyPI policy blocks new project `wet`); MCP server is a secondary surface: run `wet` with no subcommand.\n\nmcp-name: io.github.n24q02m/wet\n\n> **Renamed:** the repo, CLI and Python module are now `wet`. The PyPI distribution stays **`wet-mcp`** (install `pip install wet-mcp` / `uvx wet-mcp`). The MCP server remains available: run `wet` with no subcommand (bare = MCP passthrough); the legacy `wet-mcp` console alias still works.\n\n**Open-source MCP server for AI agents: web search, content extraction, and library docs.**\n\n| Phase | Status | Scope |\n|---|---|---|\n| Phase 1 | Shipped | web-core ScrapingAgent migration, smart chunks output, search polish, media slim |\n| Phase 2 | Shipped | Context7-level docs search: library index (Tier 1 + Tier 2), version-aware queries with token cap, project lock (Cabinets) |\n| Phase 3 | **Shipped** | `extract.agent` multi-step research with cited synthesis, `extract.interact` click/fill/submit via patchright (optional session persistence), `docs_004_chunk_summaries` migration, **`media.analyze` removed (v2.0.0)** |\n\n> **Current release: v3.x.** `media(action=\"analyze\")` was removed in the\n> v2.0.0 BREAKING release. Use\n> [`imagine-mcp`](https://github.com/n24q02m/imagine-mcp)'s\n> `understand` action for vision/audio/video analysis. See\n> [`docs/migration.md`](docs/migration.md) for the upgrade recipe.\n\n\n[![Mode](https://img.shields.io/badge/mode-daemon_%C2%B7_http_remote_relay-5C6BC0)](https://mcp.n24q02m.com/get-started/modes-overview/)\n[![CI](https://github.com/n24q02m/wet/actions/workflows/ci.yml/badge.svg)](https://github.com/n24q02m/wet/actions/workflows/ci.yml)\n[![codecov](https://codecov.io/gh/n24q02m/wet/graph/badge.svg?token=JK19TRLPEX)](https://codecov.io/gh/n24q02m/wet)\n[![PyPI](https://img.shields.io/pypi/v/wet-mcp?logo=pypi&logoColor=white)](https://pypi.org/project/wet-mcp/)\n[![License: Apache-2.0](https://img.shields.io/github/license/n24q02m/wet)](LICENSE)\n\n\n[![Python](https://img.shields.io/badge/Python-3776AB?logo=python&logoColor=white)](#)\n[![SearXNG](https://img.shields.io/badge/SearXNG-3050FF?logo=searxng&logoColor=white)](#)\n[![MCP](https://img.shields.io/badge/MCP-000000?logo=anthropic&logoColor=white)](#)\n[![semantic-release](https://img.shields.io/badge/semantic--release-e10079?logo=semantic-release&logoColor=white)](https://github.com/python-semantic-release/python-semantic-release)\n[![Renovate](https://img.shields.io/badge/renovate-enabled-1A1F6C?logo=renovatebot&logoColor=white)](https://developer.mend.io/)\n\n\n
\n Sister projects from n24q02m (click to expand)\n\n| Project | Tagline | Tag |\n|---|---|---|\n| [agent-chat-plugin](https://github.com/n24q02m/agent-chat-plugin) | Peer AI agents chat in a shared folder — no human relay, no orchestrator, wor... | Tooling |\n| [crg](https://github.com/n24q02m/crg) | Knowledge graph for token-efficient code reviews -- semantic search and call-... | MCP |\n| [better-drive](https://github.com/n24q02m/better-drive) | 2-way Google Drive sync with .driveignore filter — rclone engine, Windows tray | Tooling |\n| [better-email-mcp](https://github.com/n24q02m/better-email-mcp) | IMAP/SMTP email for AI agents -- read, send, organize folders, and manage att... | MCP |\n| [better-godot-mcp](https://github.com/n24q02m/better-godot-mcp) | Composite MCP server for Godot Engine -- 17 composite tools for AI-assisted g... | MCP |\n| [better-notion-mcp](https://github.com/n24q02m/better-notion-mcp) | Markdown-first Notion for AI agents -- pages, databases, blocks, and comments... | MCP |\n| [better-semantic-release](https://github.com/n24q02m/better-semantic-release) | Drop-in python-semantic-release fork with built-in release-safety guards (orp... | Tooling |\n| [better-telegram-mcp](https://github.com/n24q02m/better-telegram-mcp) | Telegram for AI agents -- messages, chats, media, and contacts across both bo... | MCP |\n| [better-workspace-mcp](https://github.com/n24q02m/better-workspace-mcp) | Google Workspace MCP server (Docs/Drive/Calendar/Gmail/Sheets/Slides/Tasks/Ch... | MCP |\n| [claude-plugins](https://github.com/n24q02m/claude-plugins) | Claude Code plugin marketplace for the n24q02m MCP servers -- install web sea... | Marketplace |\n| [imagine-mcp](https://github.com/n24q02m/imagine-mcp) | Image and video understanding + generation for AI agents -- across Gemini, Op... | MCP |\n| [jules-task-archiver](https://github.com/n24q02m/jules-task-archiver) | Chrome Extension for bulk operations on Jules tasks via batchexecute API -- a... | Tooling |\n| [mcp-core](https://github.com/n24q02m/mcp-core) | Shared foundation for building MCP servers -- Streamable HTTP transport, OAut... | MCP |\n| [mnemo](https://github.com/n24q02m/mnemo) | Persistent AI memory with hybrid search and embedded sync. Open, free, unlimi... | MCP |\n| [fastretrieval](https://github.com/n24q02m/fastretrieval) | Multi-model embedding and reranking runtime via ONNX and GGUF | Library |\n| [skret](https://github.com/n24q02m/skret) | Secrets without the server. | CLI |\n| [tacet](https://github.com/n24q02m/tacet) | A self-distilling neuro-symbolic cascade that amortises LLM cost across knowl... | Tooling |\n| [web-core](https://github.com/n24q02m/web-core) | Shared web infrastructure package for search, scraping, HTTP security, and st... | Library |\n| [wet](https://github.com/n24q02m/wet) | Open-source MCP server for AI agents: web search, content extraction, and lib... | MCP |\n\n
\n\n\n## Table of contents\n\n- [Features](#features)\n- [Status](#status)\n- [Quick install](#quick-install)\n- [Self-host usage](#self-host-usage)\n- [Configuration](#configuration)\n- [Documentation](#documentation)\n- [Tools](#tools)\n- [CLI](#cli)\n- [Comparison](#comparison)\n- [Security](#security)\n- [Build from Source](#build-from-source)\n- [Deploy to Cloudflare](#deploy-to-cloudflare)\n- [Smithery](#smithery)\n- [Trust Model](#trust-model)\n- [License](#license)\n\n\n\n\n \"WET\n\n\n## Features\n\n- **Web Search** -- Embedded SearXNG metasearch (Google, Bing, DuckDuckGo, Brave) with query expansion, TTL cache (1 h general / 5 min time-sensitive), standardized citation format, and 200-token snippet cap. Optional cloud search backends (Tavily, Brave, Exa, Kagi, OpenRouter, Firecrawl) as a fallback chain via `SEARCH_BACKENDS`\n- **Academic Research** -- Search Google Scholar, Semantic Scholar, arXiv, PubMed, CrossRef, BASE\n- **Library Docs** -- Auto-discover and index documentation with FTS5 hybrid search, HyDE-enhanced retrieval, and version-specific docs\n- **Content Extract** -- 5-strategy escalation chain via `n24q02m-web-core` `ScrapingAgent` (`basic_http` -> `tls_spoof` -> render backends from `BROWSER_BACKENDS` (`native` / `browserless` / `cf-browser-rendering`) -> optional key-gated `captcha`), markitdown bridge for low-tier HTML/MD fallback, smart chunks structured output (clean text + markdown + JSON-LD + code blocks + metadata), batch processing (up to 50 URLs), deep crawling, site mapping\n- **Local File Conversion** -- Convert PDF, DOCX, XLSX, CSV, HTML, EPUB, PPTX to Markdown\n- **Media** -- List + download images / videos / audio files. `analyze` was removed in v2.0.0 -- use `imagine-mcp.understand` for vision/audio inference\n- **Anti-bot** -- Stealth strategies bypass Cloudflare, Medium, LinkedIn, Twitter\n- **Zero Config** -- Built-in local reference embedding + reranking through fastretrieval, no API keys needed. Optional cloud models configured per task via the `[models.embed|rerank|chat|jev_score]` cells in `~/.wet/config.toml` (OpenRouter default; one `OPENROUTER_API_KEY` serves every cell)\n- **Sync** -- Cross-machine sync of indexed docs via Google Drive (OAuth Device Code, no browser redirect)\n\n## Quick install\n\n```bash\n# Method 1 (default): plugin install via Claude Code\n/plugin marketplace add n24q02m/claude-plugins\n/plugin install wet@n24q02m-plugins\n\n# Method 2 (CLI): direct uvx invocation\nclaude mcp add wet -- uvx wet-mcp\n\n# Method 3 (source-built container for HTTP / multi-device / OAuth)\ndocker build --target http -t wet:local .\ndocker run -d --name wet-http -p 8084:8080 \\\n -v wet-data:/data -e PUBLIC_URL=https://wet.example.com \\\n wet:local\n\n# Method 4 (remote): point a client at an HTTP deployment\nclaude mcp add --transport http wet https:///mcp\n```\n\nInstall matrix (stdio unless noted; see the [Setup](https://mcp.n24q02m.com/servers/wet/setup/) page for full steps):\n\n| Client | Install |\n|---|---|\n| Claude Code (plugin) | `/plugin marketplace add n24q02m/claude-plugins` then `/plugin install wet@n24q02m-plugins` |\n| Claude Code (stdio) | `claude mcp add wet -- uvx wet-mcp` |\n| Codex | register stdio command `uvx wet-mcp` under `mcp_servers` in `~/.codex/config.toml` |\n| Gemini CLI | add the `mcpServers` JSON below to `~/.gemini/settings.json` |\n| Cursor / Windsurf | add the `mcpServers` JSON below via the client's MCP settings (`mcp.json`) |\n| Any client (HTTP self-host) | point the client at `https:///mcp` (Streamable HTTP, OAuth-gated) |\n\nPublic OCI image publication is discontinued. Existing historical registry tags\nremain untouched; new container deployments build from source or use the\nCloudflare-managed registry.\n\nThe HTTP endpoint speaks Streamable HTTP and is OAuth-gated -- your client is\nprompted to authenticate in the browser on first connect (no API key to paste).\nStand one up via Method 3 or the\n[Deploy to Cloudflare](#deploy-to-cloudflare) section.\n\nFull setup matrices live at the canonical docs site\n[mcp.n24q02m.com/servers/wet/setup/](https://mcp.n24q02m.com/servers/wet/setup/)\nand the paste-to-agent snippets at\n[claude-plugins/plugins/wet/setup-with-agent.md](https://github.com/n24q02m/claude-plugins/blob/main/plugins/wet/setup-with-agent.md)\n(per Spec F single source of truth).\n\n## Self-host usage\n\nTwo supported ways to run the always-listening HTTP server. In both, the MCP\nendpoint is `http://127.0.0.1:8000/mcp` (Streamable HTTP) and the config root\nis `~/.wet/` (`config.toml`, `docs.db`, `subs/`). Never run the server as a\nspawned subprocess of a client — register the endpoint in your client instead.\n\n### Dev (uv, no-auth loopback)\n\n```bash\ngit clone https://github.com/n24q02m/wet && cd wet\nuv run wet config init # writes ~/.wet/config.toml (default: auth = \"no-auth\")\nuv run wet # serves http://127.0.0.1:8000/mcp\n```\n\n`no-auth` refuses non-loopback binds — the server is localhost-only until you\nswitch to token auth.\n\n### Always-on (docker)\n\n```bash\n# 1. Mint a token and its scrypt hash (hull-core canonical; the hash command\n# never echoes the token itself)\nopenssl rand -hex 32 # the token — give it to clients, keep it secret\nuv run wet token hash # paste the output as token_hash\n\n# 2. Instance config from the example\ncp docker-config/config.example.toml docker-config/config.toml\n# edit [server] token_hash and the [models.*] cells (see below)\n\n# 3. Data continuity (optional): carry an existing instance over by copying\n# docs.db and subs/ into the wet-home named volume BEFORE first start:\ndocker volume create wet-wet-home && \\\n docker cp ~/.wet/docs.db wet-wet-home:/docs.db && \\\n docker cp ~/.wet/subs wet-wet-home:/subs # adjust: files land as root, chown 999:999\n\ndocker compose up -d --build\n```\n\nCompose publishes `127.0.0.1:${WET_PORT:-8000}` → container `8000` (loopback\nonly) and mounts `docker-config/config.toml` read-only at\n`/home/appuser/.wet/config.toml`. Persistent data (`docs.db`, `subs/`) lives in\nthe `wet-home` named volume; the download/cache dir in `wet-data`. Editing the\nconfig takes effect on `docker compose restart`.\n\n### Consumers\n\n```bash\n# CLI (same token, JSON-RPC over HTTP not needed — use the installed CLI):\nuvx wet-mcp --help # or `wet ` for direct tool calls\n\n# MCP client (Claude Code):\nclaude mcp add --transport http wet http://127.0.0.1:8000/mcp \\\n --header \"Authorization: Bearer \"\n\n# Raw probe:\ncurl -s -o /dev/null -w '%{http_code}\\n' http://127.0.0.1:8000/mcp \\\n -X POST -H 'Content-Type: application/json' \\\n -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{}}' # 401\ncurl -s -o /dev/null -w '%{http_code}\\n' http://127.0.0.1:8000/mcp \\\n -X POST -H 'Authorization: Bearer ' -H 'Content-Type: application/json' \\\n -H 'Accept: application/json, text/event-stream' \\\n -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"2025-03-26\",\"capabilities\":{},\"clientInfo\":{\"name\":\"curl\",\"version\":\"0\"}}}' # 200\n```\n\n### Config: local vs cloud models\n\nThe `[models.embed|rerank|chat|jev_score]` cells in `~/.wet/config.toml` are\nper-task (each its own `base_url` + `api_key` + `model`, OpenAI-spec). The\nexample ships OpenRouter examples; any OpenAI-compatible endpoint works —\ncloud, or a local server such as Ollama (`base_url =\n\"http://host.docker.internal:11434/v1\"` from the container). Keys in the file\nare host-only material; alternatively leave `api_key = \"\"` and inject at start\nvia `HULL_EMBED_API_KEY` / `HULL_RERANK_API_KEY` / `HULL_CHAT_API_KEY` /\n`HULL_JEV_SCORE_API_KEY`.\n\n## Configuration\n\nwet runs zero-config out of the box: web search uses an embedded local SearXNG,\nand embedding/reranking fall back to the bundled local ONNX models through\nfastretrieval when no cloud keys are set. For higher-quality results, configure\nthe per-task provider cells in `~/.wet/config.toml`. Operational settings are\nplain environment variables (no app prefix) -- in the HTTP self-host mode they\nare entered through the browser setup form instead.\n\n**Provider cells** -- `[models.embed|rerank|chat|jev_score]` each take their own\n`base_url` + `api_key` + `model` (OpenAI-spec). OpenRouter is the pre-wired\ndefault for every cell; any OpenAI-compatible endpoint works. Cell keys may be\nleft empty in the file and injected at start via `HULL_EMBED_API_KEY` /\n`HULL_RERANK_API_KEY` / `HULL_CHAT_API_KEY` / `HULL_JEV_SCORE_API_KEY`. When a\ncell keeps the OpenRouter default `base_url`, `OPENROUTER_API_KEY` in the\nenvironment serves it as a last-resort credential — one OpenRouter key can\npower every task. Empty embed/rerank cells fall back to the local ONNX models;\nan unkeyed chat cell disables LLM features (e.g. `extract(action=\"agent\")`).\n\n`FASTRETRIEVAL_CACHE_PATH` controls the local model cache.\n\n**Search backends** -- `SEARCH_BACKENDS` is an ordered runtime fallback chain:\n`searxng` (default, local or external via `SEARXNG_URL`), keyed `tavily` / `brave` /\n`exa` / `kagi` / `openrouter`, optional-key `firecrawl`, and credential-free `duckduckgo` /\n`startpage`. Keyed providers use `TAVILY_API_KEY`, `BRAVE_API_KEY`, `EXA_API_KEY`,\n`KAGI_API_KEY`, or `OPENROUTER_API_KEY` (comma-separate for rotation). The\n`openrouter` backend runs the query through a chat completion with the\n`openrouter:web_search` server tool and maps the returned `url_citation`\nannotations onto the shared result shape; `OPENROUTER_MODEL` (default a\nfree-tier model), `OPENROUTER_BASE_URL`, and `OPENROUTER_SEARCH_ENGINE`\noverride its behavior. Firecrawl attempts a keyless request when `FIRECRAWL_API_KEY`\nis absent; rejection or a DuckDuckGo/Startpage bot challenge advances the chain.\n\nResult reranking uses the configured rerank cell ([models.rerank] — cloud via\nOpenRouter by default, or local ONNX when unkeyed); the same chain serves web\nsearch, research, similar-page search, agent search, and docs\ndiscovery/indexing fallbacks. SearXNG retains its science-category\nfilter for research; other providers use their own search capabilities.\nHosted users configure the chain and keys in their own relay record. An empty\nhosted record never inherits an operator's provider key or local SearXNG URL;\nsingle-user stdio still uses env/settings and preserves the public local path.\n\n**Browser render backends** -- `BROWSER_BACKENDS` (CSV, escalation chain) picks\nthe headless render leg of `extract`: `native` (in-process chromium, the\nzero-config default), `browserless` (self-host render service -- set\n`BROWSERLESS_URL` + `BROWSERLESS_TOKEN`), and `cf-browser-rendering` (Cloudflare\nBrowser Rendering -- set `CF_ACCOUNT_ID` + `CF_BROWSER_RENDERING_TOKEN`). Empty\nchain falls back to `native`. Set `CAPSOLVER_API_KEY` to append an optional,\nkey-gated CAPTCHA tier as the last escalation step.\n\n**Robots policy** -- set `RESPECT_ROBOTS_TXT=true` to enforce `robots.txt`\nacross both the `extract` strategy chain and the Crawl4AI-backed `crawl`,\n`sitemap`, and `list_media` actions. The default is `false` to preserve existing\ndeployment behaviour; configure this process-level policy explicitly when the\noperator requires robots enforcement.\n\n**Invisible tier + seeded identity** (opt-in) -- append `invisible` to\n`BROWSER_BACKENDS` to add a stealth-Firefox engine tier as the last escalation\nstep (`hull-core[invisible]` extra: `pip install \"wet-mcp[invisible]\"`). One\ncoherent browser identity spans the whole chain: seed it with `WET_IDENTITY_SEED`\n(or let wet derive once and persist it under `~/.wet/subs//identity.json`)\nand install the `wet[identity]` extra. Without the extras the chain keeps\nlegacy behaviour (warn-once). `STEALTHFOX_BINARY` points at a patched Firefox\nbuild to skip the engine download; `IDENTITY_PROFILE_DIR` (default\n`~/.wet/subs//profiles`) keeps persistent browser profiles per namespace.\n\n**Disable local fallbacks** -- opt out of the heavy in-process local fallbacks\nper capability (e.g. on a slim container that renders/searches/embeds via cloud\nbackends only): `DISABLE_LOCAL_BROWSER`, `DISABLE_LOCAL_SEARCH`,\n`DISABLE_LOCAL_EMBED`, `DISABLE_LOCAL_RERANK`.\n\n**Docs sync** -- `SYNC_ENABLED` (default `true`), `GOOGLE_DRIVE_CLIENT_ID`\n(required for sync), `SYNC_FOLDER` (default `wet`), `SYNC_INTERVAL` (default\n`300`s). Sync uses Google Drive over the OAuth Device Code flow (no browser\nredirect).\n`DOCS_DB_BACKEND=cf-d1` disables GDrive/S3 file sync, including automatic\nstartup, relay wizard and device-code setup, even if legacy sync settings remain.\nNon-CF SQLite deployments retain GDrive sync; `SYNC_S3_BUCKET` selects S3\ninstead, and `SYNC_ENABLED=false` disables both.\n\n**HTTP self-host** -- `MCP_TRANSPORT=http`, `PUBLIC_URL=`. The setup\nform is gated by `MCP_RELAY_PASSWORD`; multi-user deployments require\n`CREDENTIAL_SECRET` (per-user vault key), `MCP_JWT_SIGNING_SECRET` (rotatable\nOAuth JWT key), and `MCP_DCR_SERVER_SECRET`.\n\nExample stdio config (one OpenRouter key powers every provider cell):\n\n```json\n{\n \"mcpServers\": {\n \"wet\": {\n \"command\": \"uvx\",\n \"args\": [\"wet-mcp\"],\n \"env\": {\n \"OPENROUTER_API_KEY\": \"sk-or-xxx\"\n }\n }\n }\n}\n```\n\nTo use a different endpoint or model per task, edit the `[models.*]` cells in\n`~/.wet/config.toml`, or inject `HULL_EMBED_API_KEY` / `HULL_RERANK_API_KEY` /\n`HULL_CHAT_API_KEY` / `HULL_JEV_SCORE_API_KEY` at start instead of storing keys\nin the file.\n\n## Status\n\nStable architecture with two transports: **stdio** (default, local) and\n**HTTP** (self-host, OAuth-gated). No daemon-bridge layer and no auto-spawn\nfrom stdio. The `media.analyze` action was removed in the v2.0.0 BREAKING\nrelease -- see [`docs/migration.md`](docs/migration.md) for the upgrade\nrecipe. Current release line: v3.x.\n\n## Documentation\n\nFull docs at **[mcp.n24q02m.com/servers/wet/setup/](https://mcp.n24q02m.com/servers/wet/setup/)**:\n\n- [Setup](https://mcp.n24q02m.com/servers/wet/setup/) -- install methods for Claude Code, Codex, Gemini CLI, Cursor, Windsurf, mcp.json\n- [Modes overview](https://mcp.n24q02m.com/get-started/modes-overview/) -- stdio / local-relay / remote-relay / remote-oauth\n- [Multi-user setup](https://mcp.n24q02m.com/get-started/multi-user/) -- per-JWT-sub credential model\n\nIn-repo references (Spec F single source of truth: setup docs live in\n[claude-plugins/plugins/wet/](https://github.com/n24q02m/claude-plugins/tree/main/plugins/wet)):\n\n- [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) -- web-core ScrapingAgent integration, strategy chain, storage layout, LLM provider dispatch\n- [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) -- v1.x baseline coverage / latency placeholders + tier-1 fixture metrics\n\n**Install with AI agent** -- paste this to your AI coding agent:\n\n> Install MCP server `wet` following the steps at\n> https://raw.githubusercontent.com/n24q02m/claude-plugins/main/plugins/wet/setup-with-agent.md\n\n## Tools\n\n6 MCP tools (3 domain + `config` + `help` + `config__open_relay`). The legacy\n`setup` tool merged into `config` action dispatch.\n\n| Tool | Description |\n|:-----|:------------|\n| `search` | Web (SearXNG metasearch), news, images, academic research (Scholar / arXiv / PubMed / CrossRef / Semantic Scholar / BASE), library docs (HyDE + FTS5), find similar pages. Includes `docs_resolve` (library name -> ranked id), `docs_query` (version-aware + topic + 5000-token cap), `docs_lock_project` (Cabinets project pin via pyproject / package.json / go.mod / Cargo.toml manifest detection). |\n| `extract` | URL -> smart chunks dict (`clean_text` + `markdown` + `structured_data` + `code_blocks` + `metadata`) via web-core 5-strategy chain. Batch processing (up to 50 URLs), deep crawling, site mapping, local file conversion (PDF/DOCX/XLSX/PPTX/EPUB), structured extraction (JSON Schema) |\n| `media` | `list` (discover URLs from gallery pages), `download` (SSRF-safe). `analyze` was removed in v2.0.0 -- use `imagine-mcp.understand` instead |\n| `config` | `status`, `set`, `cache_clear`, `docs_reindex`, `warmup`, `setup_sync`, `setup_status`, `setup_skip`, `setup_reset`, `setup_complete` |\n| `help` | Per-tool documentation: `search`, `extract`, `media`, `config` |\n| `config__open_relay` | Re-trigger the zero-config relay setup flow (prints a fresh relay URL for the browser form). Registered via `mcp-core`'s `register_open_relay_tool` so an LLM can restart setup without a manual restart. |\n\n> **Media boundary**: For vision / audio understanding (image captioning,\n> OCR, audio transcription, video summarization), use\n> [imagine-mcp](https://github.com/n24q02m/imagine-mcp). `media.analyze`\n> was removed in wet v2.0.0 -- use `imagine-mcp.understand` instead.\n\n## CLI\n\nThe package installs two console scripts: **`wet`** (primary) and `wet-mcp`\n(legacy alias kept so existing `uvx wet-mcp` configs keep working). A bare\ninvocation (or any leading-dash flag) starts the server; a leading positional\nargument is dispatched as a subcommand.\n\n```bash\nuvx --from wet-mcp wet warmup # try a subcommand without a persistent install\n\nwet # start the server over stdio (default transport)\nwet --http # start the server over Streamable HTTP (self-host mode)\n\nwet auth google # authorize the Google credential provider for Drive sync\nwet logout # clear the local Google Drive sync token\nwet warmup # pre-download local models + run auto-setup (SearXNG, browser) to avoid first-run delays\nwet docs reindex # drop the cached docs index for ; the next docs search re-indexes it\n```\n\n`auth google` accepts an optional bring-your-own OAuth client via `--client-id`\nand `--client-secret` (single-user / local machine only; the token is written to\nthe local store). Each subcommand prints a JSON result and exits.\n\n| Capability | wet | Brave Search | Tavily | Firecrawl | Context7 |\n|---|---|---|---|---|---|\n| Web search | Yes (SearXNG aggregation) | Yes | Yes | No | No |\n| Extract URL | Yes (5-strategy chain) | No | Yes (basic) | Yes | No |\n| Media list / download | Yes | No | No | No | No |\n| Library docs search | Yes (Tier 1 curated + Tier 2 on-demand, version-aware, Cabinets) | No | No | No | Yes |\n| Academic research | Yes (6 providers) | No | No | No | No |\n| Self-hostable | Yes | No | No | No | Yes |\n| Free tier | Yes (open source) | Limited | Limited | Limited | Yes |\n\n## Security\n\n- **SSRF prevention** -- URL validation on crawl targets\n- **Graceful fallbacks** -- Cloud → Local embedding, multi-tier crawling\n- **Error sanitization** -- No credentials in error messages\n- **File conversion sandboxing** -- Optional `CONVERT_ALLOWED_DIRS` restriction\n\n## Build from Source\n\n```bash\ngit clone https://github.com/n24q02m/wet.git\ncd wet\nuv sync\nuv run wet\n```\n\n## Deploy to Cloudflare\n\n[![Deploy to Cloudflare](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/n24q02m/wet)\n\nRun your own single-user wet instance serverless on Cloudflare (Containers + D1 + Vectorize + KV).\n\n**Prerequisites:** a Cloudflare account on the **Workers Paid plan** — required for Containers, D1, and Vectorize (the Cloudflare free tier does not include them) — and the `wrangler` CLI.\n\n1. `git clone https://github.com/n24q02m/wet && cd wet`\n2. `wrangler login`\n3. Provision resources and apply the D1 schema:\n ```\n wrangler d1 create wet-docs\n wrangler d1 execute wet-docs --file migrations/0001_init_wet.sql --remote\n wrangler d1 execute wet-docs --file migrations/0002_project_context.sql --remote\n wrangler d1 execute wet-docs --file migrations/0003_version_index_state.sql --remote\n wrangler vectorize create wet-docs-vectors --dimensions 768 --metric cosine\n wrangler kv namespace create wet-kv\n ```\n Paste the returned IDs into `wrangler.jsonc`.\n4. Build the slim HTTP image from this checkout and push it directly to\n Cloudflare's managed registry (CF Containers cannot pull from external\n registries):\n ```\n docker build --target http --build-arg SLIM=1 -t wet:beta .\n wrangler containers push wet:beta # prints registry.cloudflare.com//wet:beta\n ```\n5. Set operator auth/storage and Browser Run secrets:\n ```\n wrangler secret put CREDENTIAL_SECRET\n wrangler secret put MCP_JWT_SIGNING_SECRET\n wrangler secret put MCP_RELAY_PASSWORD\n wrangler secret put MCP_DCR_SERVER_SECRET\n wrangler secret put CF_BROWSER_RENDERING_TOKEN\n ```\n6. `wrangler deploy` and complete setup in the browser relay form at your Worker domain.\n\nStorage maps to Cloudflare via `MCP_STORAGE_BACKEND=cf-kv` (credentials/tokens, encrypted),\n`DOCS_DB_BACKEND=cf-d1` (docs + BM25 full-text), and Vectorize (embeddings). The\ndefault headless renderer is `BROWSER_BACKENDS=cf-browser-rendering`. Local\nONNX fallbacks are disabled in the slim image; configure search and cloud\nretrieval through each authenticated subject's relay record. Worker-wide\nsearch/model chains and provider keys are not forwarded to the container.\n\nThe per-task `[models.*]` cells (in `config.toml`) configure cloud\nembed/rerank/chat: point each cell's `base_url` at any OpenAI-compatible\nendpoint — OpenRouter is the default — and supply the key via the cell's\n`api_key` or the matching `HULL__API_KEY` env. One `OPENROUTER_API_KEY`\nserves every cell that keeps the OpenRouter default `base_url`.\nStore keyed search-provider credentials (such as `TAVILY_API_KEY`) in the same\nsubject record. Provision Vectorize and `EMBEDDING_DIMS` for a dimension\nsupported by the selected embedding model.\n\n### Deployment (maintained instance)\n\nEvery tagged release deploys automatically (only while the `CF_DEPLOY_ENABLED`\ngate is on -- see the pause note below): the CD `deploy-cf` job checks out\nthe released tag, builds the http-slim image, pushes it to the Cloudflare-managed\nregistry as immutable `:`, deploys the Worker, and gates on a canary\nhealth check -- a release is live at exactly its own version. A beta dispatch\nredeploys the beta; a stable dispatch is maintainer-gated. Manual `wrangler deploy`\nagainst the maintained instance is not permitted: it would break the\nrelease-tag ↔ live-image correspondence. Self-hosting on your own Cloudflare\naccount (the button above) is unaffected.\n\n> **Paused 2026-09-13:** the CF deploy token was removed from the account as\n> off-manifest (process violation), so `deploy-cf` now no-ops behind the\n> `CF_DEPLOY_ENABLED` repo variable. The maintained instance stays frozen at\n> its last deployed release until a token is re-established via the documented\n> process and the variable is set to `true`.\n\n## Smithery\n\nwet ships a [`smithery.yaml`](smithery.yaml) so it can be installed and run\nthrough [Smithery](https://smithery.ai). The manifest declares a stdio start\ncommand (`uvx --python 3.13 wet-mcp`) with an empty config schema -- no config is\nrequired to start, and providers and credentials are configured at runtime via\nthe server's own config flow (see [Configuration](#configuration)).\n\n## Trust Model\n\nThis plugin implements **TC-Local** (machine-bound, single trust principal). See [mcp-core trust model](https://mcp.n24q02m.com/servers/mcp-core/trust-model/) for full classification.\n\n| Mode | Storage | Encryption | Who can read your data? |\n|---|---|---|---|\n| stdio (default) | `~/.wet/config.json` | AES-GCM, machine-bound key | Only your OS user (file perm 0600) |\n| HTTP self-host | Same as stdio | Same | Only you (admin = user) |\n\n## License\n\nApache-2.0 -- See [LICENSE](LICENSE).\n"}