Enables automated web searching via DuckDuckGo to retrieve search engine results, extract page content, and perform multi-level deep crawling.
Provides capabilities to perform Google searches, allowing for SERP parsing, full-text extraction from result pages, and deep-link crawling with automatic engine fallback.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Web Search MCPsearch for 'fastapi tutorial' using DuckDuckGo and extract page content"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Web Search MCP
High-performance web search service for LLM clients via Model Context Protocol. Powered by Camoufox stealth browser.


Features
3 Search Engines — Google, Bing, DuckDuckGo with automatic fallback
Multi-depth Scraping — SERP parsing → full-text extraction → outbound link crawling
Stealth Browser — Camoufox anti-detection Firefox (GeoIP, Humanize, Locale)
Auto-scaling Pool — Browser pool auto-scales at 80% utilization, configurable upper limit
Admin Dashboard — Search analytics, system monitoring, API key management, IP banning
API Key Auth — Built-in key generation (
wsm_prefix) with call limits and revocationDual Output — JSON and Markdown formats
REST API — Standard HTTP API alongside MCP protocol
Quick Start
Docker (Recommended)
git clone https://github.com/nicepkg/web-search-mcp.git
cd web-search-mcp
# Configure
cp .env.example .env
# Edit .env — set ADMIN_TOKEN
# Launch
docker compose up -d
# Verify
curl http://127.0.0.1:8897/healthCreate API Key & Register to Claude Code
# 1. Create an API key via Admin API
curl -X POST http://127.0.0.1:8897/admin/api/keys \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name": "claude-code", "call_limit": 10000}'
# Save the returned wsm_xxx key (only shown once)
# 2. Register MCP to Claude Code
claude mcp add-json -s user web-search-fast '{
"type": "http",
"url": "http://127.0.0.1:8897/mcp",
"headers": {"Authorization": "Bearer wsm_your-api-key-here"}
}'
# 3. Restart Claude Code sessionOr use the Admin Dashboard at http://127.0.0.1:8897/admin to create keys visually.
MCP Tools
Tool | Description | Timeout |
| Search the web, returns Markdown results | 25s |
| Fetch and extract content from a URL | 20s |
| List available engines and pool status | — |
web_search Parameters
Parameter | Type | Default | Description |
| string | required | Search keywords |
| string |
|
|
| int |
| 1=SERP only, 2=SERP+content, 3=SERP+content+outlinks |
| int |
| Max results (1-20) |
Search Depth
Depth | Behavior | Description |
| SERP parsing | Default. Extracts titles, links, snippets |
| SERP + content | Navigates to each result, extracts page content |
| SERP + content + outlinks | Also crawls outbound links from content |
Authentication Model
Token Type | Source | Access |
| Environment variable | Admin panel API (superuser) |
| Created via Admin panel | MCP / Search API |
ADMIN_TOKENhas superuser access to all endpointswsm_keys can only access MCP and search endpoints (not admin API)If no
ADMIN_TOKENis set and no API keys exist, all endpoints are open
REST API
Standard HTTP API available alongside MCP.
# GET
curl 'http://127.0.0.1:8897/search?q=python+asyncio&engine=duckduckgo&max_results=5' \
-H 'Authorization: Bearer wsm_your-key'
# POST
curl -X POST http://127.0.0.1:8897/search \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer wsm_your-key' \
-d '{"query": "python asyncio", "engine": "duckduckgo", "depth": 2, "max_results": 5}'Parameter | Type | Default | Description |
| string | required | Search keywords (1-500 chars) |
| string |
|
|
| int |
| Scraping depth: 1-3 |
| string |
|
|
| int |
| Max results (1-50) |
| int |
| Timeout in seconds (5-120) |
Admin Dashboard
Access http://127.0.0.1:8897/admin and login with ADMIN_TOKEN.
Dashboard — Search stats, CPU/memory monitoring, browser pool status, latency charts, engine distribution, success rate
API Keys — Create/revoke keys with call limits
IP Bans — Ban/unban IP addresses
Search Logs — Search history with IP filtering
Engine Status
Engine | Status | Notes |
DuckDuckGo | Stable | Recommended default, HTML-lite mode |
Limited | May trigger captcha on some IPs, auto-fallback | |
Bing | Available | Uses |
When Google is blocked, automatically falls back: DuckDuckGo → Bing.
Deployment
Endpoints
URL | Description |
| MCP endpoint (Streamable HTTP) |
| Health check |
| Admin dashboard |
| REST API |
Reverse Proxy (Nginx)
If deploying behind Nginx with HTTPS:
location / {
proxy_pass http://127.0.0.1:8897;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization $http_authorization;
# Required for MCP Streamable HTTP (SSE)
proxy_buffering off;
proxy_cache off;
proxy_http_version 1.1;
proxy_set_header Connection '';
proxy_read_timeout 120s;
proxy_send_timeout 120s;
}Cloudflare users: Add a WAF exception rule for
/mcppath, or use DNS-only mode (grey cloud).
Environment Variables
Variable | Default | Description |
| — | Admin panel auth token (superuser) |
|
| Initial browser tab count |
|
| Auto-scaling upper limit |
| — | Proxy server (socks5/http) |
| — | Target OS fingerprint (windows/macos/linux) |
| — | Custom font list |
|
| Block WebGL fingerprinting |
| — | Firefox addon paths |
|
| Server port |
|
| SQLite database path |
| — | Redis connection URL (optional) |
Development
# Install dependencies
pip install -e ".[dev]"
# Install Camoufox browser
python -m camoufox fetch
# Start server
python -m src.mcp_server --transport http --host 127.0.0.1 --port 8897
# Run tests (96 tests)
pytest tests/ -v
# Type check
mypy src/
# Lint & format
ruff check src/ --fix && ruff format src/Architecture
web-search-mcp/
├── src/
│ ├── mcp_server.py # MCP server entry (FastMCP + middleware + Admin)
│ ├── config.py # Configuration management
│ ├── api/
│ │ ├── routes.py # HTTP API routes
│ │ └── schemas.py # Pydantic request/response models
│ ├── core/
│ │ └── search.py # Search logic (shared by MCP + HTTP)
│ ├── engine/
│ │ ├── base.py # Search engine abstract base class
│ │ ├── google.py # Google (JS DOM + captcha detection)
│ │ ├── bing.py # Bing (global.bing.com)
│ │ └── duckduckgo.py # DuckDuckGo (HTML-lite)
│ ├── scraper/
│ │ ├── browser.py # BrowserPool (auto-scaling + health monitoring)
│ │ ├── parser.py # HTML content parser
│ │ └── depth.py # Multi-depth scraping
│ ├── formatter/
│ │ ├── json_fmt.py # JSON formatter
│ │ └── markdown_fmt.py # Markdown formatter
│ ├── admin/
│ │ ├── database.py # SQLite init + migrations
│ │ ├── repository.py # Data access layer
│ │ ├── routes.py # Admin REST API
│ │ └── static/ # Admin SPA build output
│ └── middleware/
│ ├── api_key_auth.py # Bearer token auth (ADMIN_TOKEN + DB keys)
│ ├── ip_ban.py # IP ban middleware
│ └── search_log.py # Search logging middleware
├── admin-ui/ # Admin frontend (React + Vite + Tailwind)
├── tests/ # Test suite (96 tests)
├── docker-compose.yml
├── Dockerfile
└── pyproject.tomlTech Stack
Component | Technology |
Web Framework | FastAPI + Uvicorn + Starlette |
MCP Framework | FastMCP (mcp >= 1.25.0) |
Browser Engine | Camoufox (anti-detection Firefox, Playwright) |
Async Runtime | asyncio + Semaphore concurrency control |
HTML Parsing | BeautifulSoup4 + lxml |
Content Conversion | markdownify (HTML → Markdown) |
Database | SQLite (aiosqlite) |
Cache | Redis (optional) |
Admin Frontend | React + Vite + Tailwind CSS + recharts |
Validation | Pydantic v2 |
功能特性
三大搜索引擎 — Google、Bing、DuckDuckGo,自动回退
多层深度抓取 — SERP 解析 → 正文提取 → 外链抓取
反检测浏览器 — Camoufox 真实浏览器指纹(GeoIP、Humanize、Locale)
自动扩容 — 浏览器池并发达 80% 时自动扩容,上限可配
Admin 管理面板 — 搜索统计、系统监控、API Key 管理、IP 封禁
API Key 认证 — 内置密钥生成(
wsm_前缀),支持调用限额和吊销双格式输出 — JSON / Markdown
REST API — 标准 HTTP API,与 MCP 协议并行提供
快速开始
Docker 部署(推荐)
git clone https://github.com/nicepkg/web-search-mcp.git
cd web-search-mcp
# 配置环境变量
cp .env.example .env
# 编辑 .env,设置 ADMIN_TOKEN
# 启动服务
docker compose up -d
# 验证
curl http://127.0.0.1:8897/health创建 API Key 并注册到 Claude Code
# 1. 通过 Admin API 创建 API Key
curl -X POST http://127.0.0.1:8897/admin/api/keys \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{"name": "claude-code", "call_limit": 10000}'
# 保存返回的 wsm_xxx 密钥(仅创建时可见)
# 2. 注册 MCP 到 Claude Code
claude mcp add-json -s user web-search-fast '{
"type": "http",
"url": "http://127.0.0.1:8897/mcp",
"headers": {"Authorization": "Bearer wsm_your-api-key-here"}
}'
# 3. 重启 Claude Code 会话也可以访问 http://127.0.0.1:8897/admin 通过 Admin 面板可视化创建密钥。
本地 stdio 模式
无需 Docker,Claude Code 直接通过 stdin/stdout 通信:
pip install -e .
python -m camoufox fetch
claude mcp add-json -s user web-search-fast '{
"type": "stdio",
"command": "python",
"args": ["-m", "src.mcp_server", "--transport", "stdio"],
"env": {"PYTHONUNBUFFERED": "1"},
"cwd": "'$(pwd)'"
}'认证模型
Token 类型 | 来源 | 访问范围 |
| 环境变量 | Admin 面板 API(超级权限) |
| Admin 面板创建 | MCP / 搜索 API |
ADMIN_TOKEN拥有所有端点的超级权限wsm_密钥只能访问 MCP 和搜索端点(不能访问 Admin API)如果未配置
ADMIN_TOKEN且无 API Key,所有端点开放访问
反向代理注意事项
使用 Nginx 反向代理时,MCP Streamable HTTP 需要关闭缓冲:
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Connection '';
proxy_read_timeout 120s;Claude Code 如何默认使用这个工具进行搜索
编辑
~/.claude/CLAUDE.md添加下面的内容
## Web Search
* 优先使用 `web-search-fast`
Cloudflare 用户:需要为
/mcp路径添加 WAF 例外规则,或使用 DNS-only 模式(灰色云朵)。