mcp-search

MCP Search

Containerized search and web scraping infrastructure powered by Model Context Protocol

Overview

MCP Search is a containerized infrastructure that provides web search and URL scraping capabilities through the Model Context Protocol (MCP). It combines SearXNG (privacy-respecting metasearch engine) and Firecrawl (web scraping API) with their respective MCP servers, backed by Redis, PostgreSQL, RabbitMQ, and Playwright.

Architecture

graph TD
    MCP["MCP Client"] --> mcp_sx["mcp-searxng :30019"]
    MCP --> mcp_fc["mcp-firecrawl :30028"]
    mcp_sx --> searxng["SearXNG :8080"]
    mcp_fc --> firecrawl["Firecrawl :3000"]
    firecrawl --> playwright["Playwright :3000"]
    firecrawl --> nuq_pg["nuq-postgres :5432"]
    firecrawl --> rabbitmq["RabbitMQ :5672"]
    firecrawl --> redis["Redis :6379"]

Services

Service	Image	Port	Description
searxng	searxng/searxng	8080	Privacy-respecting metasearch engine (web UI available)
mcp-searxng	isokoliuk/mcp-searxng	30019	MCP server for SearXNG API
firecrawl	ghcr.io/firecrawl/firecrawl	3000	Web scraping and crawling API
mcp-firecrawl	mcp/firecrawl	30028	MCP server for Firecrawl API
redis	redis:7-alpine	6379	Cache and rate limiting
nuq-postgres	ghcr.io/firecrawl/nuq-postgres	5432	PostgreSQL for Firecrawl
playwright	mcr.microsoft.com/playwright	3000	Headless browser for JS rendering
rabbitmq	rabbitmq:3-management	5672 / 15672	Message broker for Firecrawl

Usage

Requirements

Dependency	Minimum Version
Docker or Podman (with compose)	23.0 / 5.0
Bash	5.0
OpenSSL	3.x

Quick Start

1. Clone the repository:

git clone https://github.com/Mournweiss/mcp-search.git
cd mcp-search

2. Run the build script:

chmod +x build.sh
./build.sh

The script will:

• Create .env from .env.example if it doesn't exist
• Sync missing variables from the template
• Generate secrets (SEARXNG_SECRET_KEY, POSTGRES_PASSWORD, etc.)
• Build and start all services in detached mode

Arguments

--podman, -p          Use podman-compose instead of docker-compose
--docker, -d          Use docker-compose explicitly
--foreground, -f      Run in foreground mode (attach logs)
--no-keygen, -n       Skip secret generation
--help, -h            Show help message

3. Connect to LLM:

   Both MCP servers run as streamable HTTP endpoints.

   Server	Endpoint	Transport
   mcp-searxng	http://localhost:30019/mcp	Streamable HTTP
   mcp-firecrawl	http://localhost:30028/mcp	Streamable HTTP

Environment Variables

All configuration is managed through .env. See .env.example for the full list of variables.

Primary variables:

Variable	Default	Description
PROJECT_NAME	mcp-search	Docker Compose project prefix
LOG_LEVEL	info	Global log level
SEARXNG_PORT	8080	SearXNG HTTP port
MCP_SEARXNG_PORT	30019	MCP SearXNG server port
FIRECRAWL_PORT	3000	Firecrawl API port
MCP_FIRECRAWL_PORT	30028	MCP Firecrawl server port
RABBITMQ_PORT	5672	RabbitMQ port
RABBITMQ_MANAGEMENT_PORT	15672	RabbitMQ management UI port
SEARXNG_ENGINES	google,duckduckgo,bing	Search engines to enable
SEARXNG_CATEGORIES	general,it,science	Search categories

License

This project uses AGPL-3.0 licensed components. See individual repositories for their respective licenses:

• SearXNG - AGPL-3.0
• mcp-searxng - MIT
• Firecrawl - AGPL-3.0
• firecrawl-mcp-server - MIT
• Redis - BSD
• Playwright - Apache-2.0
• RabbitMQ - MPL-2.0