Web Search MCP

A comprehensive, production-ready research server for the Model Context Protocol (MCP). Provide your LLM clients with real-time access to the web and more.

✨ Features

• 🌐 Deep Web Search: Text and news search via DuckDuckGo.

• 📄 Content Extraction: Read clutter-free full text from any URL using trafilatura. Supports multiple output formats (text, markdown, JSON), metadata extraction, and content filtering.

• 🛡️ Bot Detection Bypass: Automatic fallback to Chrome TLS impersonation when sites block requests (Cloudflare, etc.).

• 💻 Technical Docs: Targeted search for developer documentation (Python, React, etc.).

🚀 Quick Start

Installation

Install directly using uv:

uv tool install git+https://github.com/sydasif/web-search-mcp.git

Configuration

Add the server to your MCP client configuration (e.g., claude_desktop_config.json). You can optionally configure rate limits via environment variables to avoid DuckDuckGo blocking.

{
  "mcpServers": {
    "web-search": {
      "command": "web-search-mcp",
      "env": {
        "SEARCH_MCP_RATE_LIMIT_SEARCH": "30",
        "SEARCH_MCP_RATE_LIMIT_FETCH": "20"
      }
    }
  }
}

Available Environment Variables:

• SEARCH_MCP_RATE_LIMIT_SEARCH: Max search requests per minute (default: 30).

• SEARCH_MCP_RATE_LIMIT_FETCH: Max page fetch requests per minute (default: 20).

Fetch Backend Options

The fetch_page tool supports three backend modes to handle sites with bot detection:

🛠️ Tool Reference

💻 Development

1. Clone the repository

   git clone https://github.com/sydasif/web-search-mcp.git
   cd web-search-mcp

2. Sync dependencies

   uv sync

3. Run tests

   # Run all tests
   uv run pytest
   
   # Run with coverage
   uv run pytest --cov=web_search_mcp

4. Linting & Formatting

   uv run ruff check .

📄 License

This project is licensed under the MIT License.