scrapingdog-mcp

scrapingdog-mcp

A Model Context Protocol server that exposes the Scrapingdog scraping APIs as tools for MCP-compatible clients (Claude Desktop, Claude Code, and others).

Communicates over stdio. Built with TypeScript and the official MCP SDK.

What it does

Nine tools, each wrapping one Scrapingdog endpoint:

Tool	Endpoint	What it returns
scrape	/scrape	Raw HTML of any URL (JS rendering, premium proxies, geotargeting, sessions, stealth, wait)
screenshot	/screenshot	Page screenshot metadata (png/jpg/webp, full-page, viewport, quality)
google_search	/google	Google organic results, ads, knowledge graph, SERP features (JSON or HTML)
bing_search	/bing/search	Bing search results (market/geo/pagination/safe-search)
duckduckgo_search	/duckduckgo/search	DuckDuckGo results (region, date filter, pagination token)
baidu_search	/baidu/search	Baidu results (Chinese-language restriction, pagination)
x_profile	/x/profile	X (Twitter) profile data (name, handle, follower counts, bio)
x_post	/x/post	X (Twitter) post (tweet) data
datacenter_proxy	(forward proxy)	Connection details + ready-to-paste curl/Python for proxy.scrapingdog.com:8081

Every tool accepts an optional api_key argument to override the configured key for that call.

Prerequisites

• Node.js ≥ 18.17 (tested on 20.x).
• A Scrapingdog API key — get one from your Scrapingdog dashboard after signing up.

Install & build

npm install
npm run build      # outputs to dist/

Provide your API key

The key is resolved in this order (first wins):

1. The api_key argument on an individual tool call.
2. The SCRAPINGDOG_API_KEY environment variable.
3. A .env file containing SCRAPINGDOG_API_KEY=... in the working directory, ~/.scrapingdog.env, or ~/.env.

Option 2 (env var) is recommended — the key never touches disk beyond your MCP client config, and nothing ends up in tool-call logs.

Configure your MCP client

Claude Desktop / Claude Code (claude_desktop_config.json)

{
  "mcpServers": {
    "scrapingdog": {
      "command": "node",
      "args": ["/absolute/path/to/scrcpy/dist/index.js"],
      "env": {
        "SCRAPINGDOG_API_KEY": "your-key-here"
      }
    }
  }
}

Install from npm (no clone needed)

Published as @0pen1/scrapingdog-mcp. Run directly with npx (no install):

npx @0pen1/scrapingdog-mcp

The server reads SCRAPINGDOG_API_KEY from the environment. Pick your CLI below for a one-line setup.

Claude Code

claude mcp add scrapingdog --scope user \
  --env SCRAPINGDOG_API_KEY=your-key-here \
  -- npx -y @0pen1/scrapingdog-mcp

--scope user makes it available in all your projects (use local or project to scope it tighter).

OpenAI Codex CLI

codex mcp add scrapingdog \
  --env SCRAPINGDOG_API_KEY=your-key-here \
  -- npx -y @0pen1/scrapingdog-mcp

This writes to ~/.codex/config.toml. The equivalent TOML block is:

[mcp_servers.scrapingdog]
command = "npx"
args = ["-y", "@0pen1/scrapingdog-mcp"]

[mcp_servers.scrapingdog.env]
SCRAPINGDOG_API_KEY = "your-key-here"

Gemini CLI

gemini mcp add scrapingdog \
  --env SCRAPINGDOG_API_KEY=your-key-here \
  -- npx -y @0pen1/scrapingdog-mcp

Cursor / Windsurf / Claude Desktop (JSON config)

For clients that use a JSON config file, add this block. Typical locations:

• Claude Desktop — ~/Library/Application Support/Claude/claude_desktop_config.json (macOS)
• Cursor — ~/.cursor/mcp.json (or .cursor/mcp.json per-project)
• Windsurf — ~/.codeium/windsurf/mcp_config.json

{
  "mcpServers": {
    "scrapingdog": {
      "command": "npx",
      "args": ["-y", "@0pen1/scrapingdog-mcp"],
      "env": { "SCRAPINGDOG_API_KEY": "your-key-here" }
    }
  }
}

Replace your-key-here with your Scrapingdog API key from the dashboard. After editing a JSON config, restart the client so it picks up the server.

Run standalone (for testing)

SCRAPINGDOG_API_KEY=your-key node dist/index.js

It speaks JSON-RPC over stdio, so you can pipe messages to it directly.

Example tool calls

scrape          { "url": "https://example.com", "dynamic": true, "country": "us" }
google_search   { "query": "best espresso machines 2026", "results": "10", "country": "us" }
bing_search     { "query": "site:github.com mcp server", "count": "20" }
duckduckgo_search { "query": "rust async runtime", "df": "m" }
baidu_search    { "query": "人工智能", "ct": 2 }
x_profile       { "profileId": "elonmusk" }
x_post          { "tweetId": "1655608985058267139" }
screenshot      { "url": "https://example.com", "fullPage": true, "format": "png" }
datacenter_proxy { "target_url": "https://httpbin.org/ip" }

How responses are handled

• JSON endpoints (search, social, etc.): the parsed JSON is returned as text so the host can reason about it directly.
• /scrape: raw HTML is returned as text.
• /screenshot: the endpoint returns binary image bytes, which can't be carried as text cleanly. The tool reports the content type and byte length, and notes how to fetch the image directly. (If you need true image delivery, extend the handler to base64-encode into an image content block.)
• Non-2xx responses: returned as MCP error results (isError: true) with the upstream status, content type, and body. Note Scrapingdog reports invalid keys as HTTP 400 with a JSON "Unauthorized request" message — the body makes this clear.

Credit costs (per request)

Scrape: 1 (rotating proxy) → 25 (JS render + premium). Google/Bing/DDG/Baidu search: ~5. Screenshot: 5. Profile/Post scrapers: 5–10. See the pricing/credit table for the full breakdown. Requests time out after 60s upstream.

Project layout

src/
  index.ts          # entry point — creates McpServer, connects stdio transport
  scrapingdog.ts    # API key resolution + shared HTTP request helper
  result.ts         # response → MCP tool-result formatting (ok / error / fromResponse)
  tools.ts          # the 9 tool definitions (schema + handler) and registration

License

MIT