Skip to main content
Glama
thehappyboy

Web Search MCP

by thehappyboy
OverviewSchemaRelated ServersScoreDiscussions
JavaScript
Hybrid

Web Search MCP

Multi-source web search MCP server with RRF fusion, 4-layer URL extraction, and provider health tracking.

Status

Phase 2: 5 providers (SearXNG/ddgr/Tavily/Exa/opencli-zh), RRF fusion, snippet cleaning (rules + optional LLM compression), search/extract/doctor tools, provider health tracking with auto-degradation.

Related MCP server: Deep Search MCP Server

Install

Add to Claude Desktop 3P config (~/Library/Application Support/Claude-3p/configLibrary/<uuid>.json):

{
  "name": "web-search",
  "source": "user",
  "transport": "stdio",
  "command": "node",
  "args": ["/Users/hades/projects/web-search-mcp/bin/cli.js"],
  "env": {
    "SEARXNG_URL": "http://localhost:18443",
    "TAVILY_API_KEY": "...",
    "EXA_API_KEY": "...",
    "FIRECRAWL_API_KEY": "...",
    "WSM_LLM_PROVIDER": "haiku",
    "ANTHROPIC_API_KEY": "..."
  },
  "toolPolicy": {
    "search": "allow",
    "extract": "allow",
    "doctor": "allow"
  }
}

Once published to npm, replace command: "node" + absolute path with command: "npx" + args: ["-y", "@wsm/cli@latest"].

Configuration

Environment variables (or .env in cwd or ~/.agents/skills/web-search/.env):

Var

Default

Required

Description

SEARXNG_URL

http://localhost:18443

no

SearXNG base URL

TAVILY_API_KEY

—

no

Enables Tavily provider

EXA_API_KEY

—

no

Enables Exa provider

FIRECRAWL_API_KEY

—

no

Enables Firecrawl in extract pipeline (layer 3)

WSM_LLM_PROVIDER

none

no

haiku for Anthropic, local for Ollama, none to disable

ANTHROPIC_API_KEY

—

no

Required when WSM_LLM_PROVIDER=haiku

WSM_LOCAL_LLM_URL

—

no

Required when WSM_LLM_PROVIDER=local

WSM_MAX_RESULTS

10

no

Default max results

WSM_TIMEOUT_MS

25000

no

Global search timeout

Providers without keys are automatically disabled. The search tool still works with whatever is available. ddgr, opencli, defuddle are auto-detected via which.

Tools

search

{
  query: string,              // required
  max_results?: number,       // 1-30, default 10
  category?: 'general'|'news'|'science',
  mode?: 'fast'|'quality',    // quality = LLM compression (top-5 results)
  providers?: string[],       // override default list (replace semantics)
}

Returns:

{
  count: number,
  took_ms: number,
  providers_used: string[],     // providers that returned ≥1 result
  providers_failed: string[],   // providers that errored OR returned []
  results: [{
    title, url, snippet,
    providers: string[],        // which sources surfaced this URL (multi-source corroboration)
    rrf_score: number,          // reciprocal rank fusion score
  }]
}

opencli-zh auto-routing: When query contains keywords like 知乎, 小红书, 雪球, B站, 微博, or patterns like 600519.SH, the opencli-zh provider routes to the corresponding adapter. Force a specific site with opencli_sites (passed via providers extension, see orchestrator).

extract

{
  url: string,                 // required
  timeout?: number,            // 5-60 seconds, default 20
}

Tries 4 layers in order until one succeeds:

  1. defuddle — fastest, pure-text sites

  2. opencli adapter — for known sites (zhihu/xiaohongshu/xueqiu/bilibili/weibo)

  3. Firecrawl — JS-rendered pages (requires FIRECRAWL_API_KEY)

  4. opencli browser — last resort, drives real Chrome

Returns:

{
  content: string,             // markdown
  source: 'defuddle'|'opencli'|'firecrawl'|'opencli-browser'|'none',
  took_ms: number,
  error?: string,              // only when source === 'none'
}

doctor

{} // no arguments

Returns provider availability + runtime health stats:

{
  providers: {
    searxng:    { available: true, url: "http://localhost:18443" },
    ddgr:       { available: true },
    tavily:     { available: true },
    exa:        { available: true },
    opencli_zh: { available: true },
    firecrawl:  { available: true },
    llm:        { available: true, provider: "haiku" },
  },
  health: {
    tavily: {
      total: 24,
      success: 15,
      fail: 9,
      success_rate: 0.625,
      last_failure: "2026-07-06T...",
      last_failure_reason: "timeout",
      recommendation: "healthy" | "insufficient" | "degraded",
    },
    // ...
  },
  version: "0.2.0",
}

When a provider's success_rate drops below 0.5 with at least 20 samples, it's automatically skipped (recommendation: degraded). Health stats are in-memory and reset on restart.

Development

npm test                  # all tests (unit + integration, 158 total)
npm run test:unit         # unit only
npm run test:integration  # integration only (needs searxng at SEARXNG_URL)
npm start                 # start stdio server

Architecture

MCP client (Claude/Cowork)
  │ JSON-RPC over stdio
  ▼
server.js ───┬── search tool ──→ search.js (orchestrator)
             ├── extract tool ─→ extract/index.js (4-layer pipeline)
             └── doctor tool ──→ health tracker snapshot

search.js:
  Provider layer (parallel, isolated failure)
    searxng | ddgr | tavily | exa | opencli-zh
  ▼
  Merger layer: normalize → dedup(URL) → RRF(k=60)
  ▼
  Cleaner layer: rules (default) | LLM compression (quality mode, top-5)
  ▼
  Health tracker records success/failure per provider

extract/index.js:
  defuddle → opencli adapter → Firecrawl → opencli browser
  (each layer returns null on failure, falls through to next)

Phase history

  • Phase 1 (15 tasks): 4 providers (searxng/ddgr/tavily/exa), RRF fusion, rule-based cleaning, stdio MCP, search tool only

  • Phase 2 (8 tasks): opencli-zh provider, LLM quality mode, 4-layer extract, doctor + health auto-degradation

License

MIT

Install Server
F
license - not found
A
quality
B
maintenance

How are these scores calculated?

Maintenance

–Maintainers
–Response time
–Release cycle
–Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Tools

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/thehappyboy/web-search-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server