Web Search MCP
Web Search MCP
多源网络搜索 MCP 服务器,具备 RRF 融合、4 层 URL 提取以及提供商健康跟踪功能。
状态
第二阶段:5 个提供商(SearXNG/ddgr/Tavily/Exa/opencli-zh)、RRF 融合、摘要清理(规则 + 可选 LLM 压缩)、search/extract/doctor 工具、包含自动降级的提供商健康跟踪。
Related MCP server: web-mcp
安装
添加到 Claude Desktop 第三方配置(~/Library/Application Support/Claude-3p/configLibrary/<uuid>.json):
{
"name": "web-search",
"source": "user",
"transport": "stdio",
"command": "node",
"args": ["/Users/hades/projects/web-search-mcp/bin/cli.js"],
"env": {
"SEARXNG_URL": "http://localhost:18443",
"TAVILY_API_KEY": "...",
"EXA_API_KEY": "...",
"FIRECRAWL_API_KEY": "...",
"WSM_LLM_PROVIDER": "haiku",
"ANTHROPIC_API_KEY": "..."
},
"toolPolicy": {
"search": "allow",
"extract": "allow",
"doctor": "allow"
}
}发布到 npm 后,将 command: "node" + 绝对路径替换为 command: "npx" + args: ["-y", "@thehappyboy/web-search-mcp@latest"]。
配置
环境变量(或当前工作目录下的 .env,或 ~/.agents/skills/web-search/.env):
变量 | 默认值 | 必填 | 描述 |
|
| 否 | SearXNG 基础 URL |
| — | 否 | 启用 Tavily 提供商 |
| — | 否 | 启用 Exa 提供商 |
| — | 否 | 在提取管道中启用 Firecrawl(第 3 层) |
|
| 否 |
|
| — | 否 |
|
| — | 否 |
|
|
| 否 | 默认最大结果数 |
|
| 否 | 全局搜索超时时间 |
没有密钥的提供商将自动禁用。搜索工具仍可使用可用的提供商。ddgr、opencli、defuddle 通过 which 自动检测。
工具
search
{
query: string, // required
max_results?: number, // 1-30, default 10
category?: 'general'|'news'|'science',
mode?: 'fast'|'quality', // quality = LLM compression (top-5 results)
providers?: string[], // override default list (replace semantics)
}返回:
{
count: number,
took_ms: number,
providers_used: string[], // providers that returned ≥1 result
providers_failed: string[], // providers that errored OR returned []
results: [{
title, url, snippet,
providers: string[], // which sources surfaced this URL (multi-source corroboration)
rrf_score: number, // reciprocal rank fusion score
}]
}opencli-zh 自动路由:当查询包含诸如 知乎、小红书、雪球、B站、微博 等关键词,或 600519.SH 等模式时,opencli-zh 提供商会路由到对应的适配器。通过 opencli_sites 强制指定特定站点(通过提供商扩展传递,参见编排器)。
extract
{
url: string, // required
timeout?: number, // 5-60 seconds, default 20
}按顺序尝试 4 层,直到某层成功:
defuddle — 最快,纯文本站点
opencli 适配器 — 适用于已知站点(知乎/小红书/雪球/B站/微博)
Firecrawl — JS 渲染页面(需要
FIRECRAWL_API_KEY)opencli 浏览器 — 最后手段,驱动真实的 Chrome
返回:
{
content: string, // markdown
source: 'defuddle'|'opencli'|'firecrawl'|'opencli-browser'|'none',
took_ms: number,
error?: string, // only when source === 'none'
}doctor
{} // no arguments返回提供商可用性及运行时健康统计:
{
providers: {
searxng: { available: true, url: "http://localhost:18443" },
ddgr: { available: true },
tavily: { available: true },
exa: { available: true },
opencli_zh: { available: true },
firecrawl: { available: true },
llm: { available: true, provider: "haiku" },
},
health: {
tavily: {
total: 24,
success: 15,
fail: 9,
success_rate: 0.625,
last_failure: "2026-07-06T...",
last_failure_reason: "timeout",
recommendation: "healthy" | "insufficient" | "degraded",
},
// ...
},
version: "0.2.0",
}当某个提供商的 success_rate 低于 0.5 且样本数至少为 20 时,该提供商将被自动跳过(建议:degraded)。健康统计保存在内存中,重启后重置。
开发
npm test # all tests (unit + integration, 158 total)
npm run test:unit # unit only
npm run test:integration # integration only (needs searxng at SEARXNG_URL)
npm start # start stdio server架构
MCP client (Claude/Cowork)
│ JSON-RPC over stdio
▼
server.js ───┬── search tool ──→ search.js (orchestrator)
├── extract tool ─→ extract/index.js (4-layer pipeline)
└── doctor tool ──→ health tracker snapshot
search.js:
Provider layer (parallel, isolated failure)
searxng | ddgr | tavily | exa | opencli-zh
▼
Merger layer: normalize → dedup(URL) → RRF(k=60)
▼
Cleaner layer: rules (default) | LLM compression (quality mode, top-5)
▼
Health tracker records success/failure per provider
extract/index.js:
defuddle → opencli adapter → Firecrawl → opencli browser
(each layer returns null on failure, falls through to next)阶段历史
第一阶段(15 项任务):4 个提供商(searxng/ddgr/tavily/exa)、RRF 融合、基于规则的清理、stdio MCP、仅
search工具第二阶段(8 项任务):opencli-zh 提供商、LLM 质量模式、4 层
extract、doctor+ 健康自动降级
许可证
MIT
Available Tools
6 toolsdoctorC
Diagnose provider availability and health.
Inputs: no named parameters.
extractA
Extract clean markdown from a URL. Tries defuddle → opencli adapter → Firecrawl → opencli browser in order.
Inputs: url (required), timeout.
searchA
Multi-source web search with RRF fusion. Providers tried in parallel: SearXNG/ddgr/Tavily/Exa/opencli-zh. Results deduplicated by URL and reranked by reciprocal rank fusion. Snippets cleaned.
Inputs: query (required), mode, category, providers, +1 more.
vertical_batchC
Up to 5 vertical queries in parallel via AnySearch.
Inputs: queries (required).
vertical_searchA
Vertical-domain structured search via AnySearch. Use for finance (real-time quotes, fundamentals), academic (papers), security (CVE), ip (patents), legal, health. Returns structured payload (field-level data, not flattened).
Inputs: query (required), domain, sub_domain, max_results, +1 more.
vertical_sub_domainsA
Discover sub-domains and required parameters for a given vertical domain. Call before vertical_search when sub_domain schema is unknown.
Inputs: domain, domains.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 6 tools
Most tools are clearly distinct: 'doctor' for health checks, 'extract' for URL content, 'search' for general search, and 'vertical_*' for domain-specific search. However, 'vertical_search' and 'vertical_batch' both perform vertical searches, differing only in batch size, which could cause slight confusion.
Naming is inconsistent: 'doctor', 'extract', 'search' are single verbs without a clear pattern, while 'vertical_batch', 'vertical_search', and 'vertical_sub_domains' use a snake_case prefix. This mix of conventions reduces predictability.
With 6 tools, the set is well-scoped for a web search and extraction server. Each tool serves a distinct purpose without being excessive or insufficient.
The tool surface covers core functionality: health checks, general search, URL extraction, and vertical domain search with discovery. Minor gaps exist, such as no tool for image or news search, but the provided set is functionally complete for the stated domain.
Maintenance
Related MCP Connectors
One MCP server for all web data. Scrape, search, crawl, and AI research in one install.
Unified 15-tool SEO, SERP, clean Markdown & GEO (LLM citation Share-of-Voice) MCP server.
Web search API and MCP server for developers building AI agents.
MCP server for Google search results via SERP API
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceMulti-engine aggregated search MCP server that combines results from 7 search engines with deduplication, relevance ranking, and web page content extraction.1MIT
- AlicenseNot gradedqualityCmaintenanceMCP server for multi-engine web search and web page fetching, supporting parallel search, content extraction, and optional LLM-powered search summarization and deep search.6MIT
- FlicenseNot gradedqualityDmaintenanceMCP server that provides web search via LLM API gateways with server-side search, returning cited answers and multi-provider fallback.-
- AlicenseAqualityCmaintenanceA self-hosted MCP server for web search with multi-engine support, combining search, URL fetching, and deep research along with proxy routing and deduplication.3MIT