🔍 web-search-plus-mcp

Give your AI app better web search and clean page reading. web-search-plus-mcp works with Claude Desktop, Cursor, NanoBot, Hermes, and other MCP apps. It searches across the services you choose, returns the original sources, and can try another service when one fails.

web-search-plus-mcp 5.1.6 (released 2026-10-30) says why a page could not be extracted (page not found, site refused, unreachable, provider does not support the site), stops paying for fallbacks when a page does not exist, and no longer blocks Firecrawl after a site it does not support. 5.1.5 fixed DonSeTch with DonSeTch 4.7.1 and newer; 5.1.4 brought compact responses by default; see the changelog.

What's new in 5.1.3

Search and page reading now return compact JSON by default, saving context by removing repeated evidence and execution diagnostics. Source titles, URLs, snippets and bounded page text stay intact. Clients that parse the full v3 contract (including observations and routing receipts) must pass detail: "full" or set the server default below. See docs/RELEASE_5_1_3.md.

Google /goto source wrappers are dropped without contacting Google by default; classic /url?q= links decode locally. Opt in to the bounded resolver with {"serper": {"resolve_redirects": true}} in config.json. SerpBase defaults to device: "mobile" (configurable as mobile or desktop under serpbase) and prefers canonical URLs. Every search provider shares a conservative corrupt-URL filter. Compact output includes positive redirects_wrapped, redirects_resolved, redirects_dropped and invalid_urls_dropped counts; full output retains their complete receipts. DonSeTch 4.10.3 is the tested version.

Earlier: 5.1.2

5.1.1 was never released, so this covers everything since 5.0.1. No provider request starts after the request deadline, web_extract returns exactly one result per requested URL in request order, and each result now carries requested_url, the URL you passed in (url is where the page was fetched). A rate limit, out-of-credit or auth error partway through a batch keeps the pages already fetched and marks the rest not tried: … instead of failing the whole batch, and URLs past the per-call limit of 10 get their own not tried row instead of disappearing. The per-request budget no longer depends on SQLite, half-open circuit probes release their lease, and the search cache is keyed by the credentials that produced a result. Brave accepts queries up to its documented 600 characters and 75 words. Hermes-only changes (profile-scoped keys, native backend, hermes commands) are not part of this package. See docs/RELEASE_5_1_2.md.

Earlier: 5.0.1

Bug fixes for 5.0.0: everyday words like "synthesizer" no longer trip the source-only filter, one bad URL no longer sinks a web_extract batch (each failed URL gets its own error), empty pages fall back to the next provider, rate-limit waits are capped at 30 s, long queries are shortened for Brave, and auto routing off or a private SearXNG URL no longer break every search. See docs/RELEASE_5_0_1.md.

Earlier: 5.0.0

Automatic routing picks providers by query type (Brave for most queries including security and shopping, Exa for docs and academic; Serper remains a later fallback), clearly detected query languages reach the provider, research mode fuses provider results, and automatic search falls back faster on errors, empty answers and slow providers. This changes which API quota is used; see CHANGELOG.md for breaking changes.

Earlier: 4.3.5

Missing-key hints now reach the MCP client, failed calls set isError: true, locale settings are applied to Brave, You.com, Firecrawl, SerpBase and SearXNG, dedup keeps ?v=-style pages apart, and web_extract stays within its size limit. See docs/RELEASE_4_3_5.md.

Earlier: 4.3.4

Missing-key errors point to web-search-plus-mcp setup, the .env file or the MCP client env block instead of an inline key in config.json, and DonSeTch no longer loses its stderr excerpt when a session closes. See docs/RELEASE_4_3_4.md.

Earlier: 4.3.3

web_extract converts internationalized hostnames such as müller.de to punycode instead of rejecting them. See docs/RELEASE_4_3_3.md.

Earlier: 4.3.2

Security hardening, matching Hermes Web Search Plus 4.3.2: stricter extract URL validation, same-origin-only redirects, response size limits, no provider error text passed on, option-like domain and URL values refused before any run, and an allowlisted DonSeTch environment. Redirects across host, port or scheme now fail and provider errors are less detailed; see docs/RELEASE_4_3_2.md.

Earlier: 4.3.1

Faster: searches run inside the MCP server instead of one Python process per call, and provider connections are reused (auto routing 1.70 s → 0.99 s in live tests). Restart the server after adding a key to .env. See 4.3.1 notes.

What's new in 4.3.0

Adaptive routing learns again: since the 3.0 engine, provider calls no longer recorded samples. provider_stats.json writes are now locked across processes, and web_search honours defaults.max_results when count is omitted. See the 4.3.0 release notes.

What's new in 4.2.1

web_search now accepts no_cache and cache_ttl, matching the CLI. The optional DonSeTch 4.2.9 adapter stays separately installed. See the 4.2.1 release notes.

What's new in 4.2.0

Optional Jev is off by default. Enable with setup --jev --jev-key-file PATH. Jev can confirm a news search type, score extract bodies, and fill language when inference returned none. It is not a search provider. Keep the key in TYPESAFE_API_KEY_FILE, never in config.json.

Existing MCP tools and setup stay the same. The matching Hermes plugin adds an optional native backend; this MCP package does not include that adapter. The Windows fcntl import fix from Hermes #129 is not ported here. See the 4.2.0 release notes.

🚀 Quick Start

# Run it without installing anything
uvx web-search-plus-mcp

# Or install it normally
pip install web-search-plus-mcp
web-search-plus-mcp

Add at least one search provider. You can start with one and add more later.

Related MCP server: OneSearch MCP Server

✨ What it does

• 15 search providers — use one service or let Web Search Plus choose

• 9 page-reading providers — turn web pages into clean text

• Automatic fallback — try another provider when the first one fails

• Real sources — keep the links and text the result came from

• Research mode — search several providers for broader questions

• Optional details — see which provider ran and how the result was found

• Simple setup tools — check your config and create a starter setup

• Optional local search — connect a separately installed DonSeTch executable through the WSP provider adapter

Version 4.0.3 tests DonSeTch 3.2.1. Version 4.0.2 adds Parallel Search modes (default fast). Version 4.0.1 hardens DonSeTch process reuse and diagnostics. Version 4.0.0 removed the optional Hound provider and added the separately installed DonSeTch 2.1.0 stdio adapter. See the 4.0.0 release notes and DonSeTch guide for migration, licensing, and runtime boundaries.

🧭 Easier onboarding

Check configured providers:

web-search-plus-mcp status

List providers or presets:

web-search-plus-mcp list providers
web-search-plus-mcp list presets

Write a starter .env template and print a canonical MCP stdio snippet:

web-search-plus-mcp setup --preset starter

The recommended starter preset is You.com + Serper + Linkup. It gives a practical source-only baseline for fast current search, Google-style discovery, and extraction workflows without wiring every provider on day one.

🧩 Agent Plugins package

This repository also contains a portable Agent Plugins 1.0 package for clients that support the standard. The root plugin.json describes the plugin and mcp.json connects the MCP server through the published PyPI package.

The portable configuration uses uvx with an exact web-search-plus-mcp version pin. Install uv first, then load this repository as an Agent Plugin in a compatible client such as VS Code, Cursor, GitHub Copilot, ChatGPT/Codex, or Kiro. Provider credentials are deliberately not stored in the plugin manifest; provide the required environment variables through the client or process environment.

The Agent Plugins format defines packaging and loading, not a central marketplace. Client-specific discovery and distribution remain outside the portable package contract.

status returns a non-zero exit code when no search provider is configured, which makes it usable as a config check in scripts.

Persistent routing preferences live in config.json rather than .env:

web-search-plus-mcp config show
web-search-plus-mcp config set-default you        # strict fixed-provider mode
web-search-plus-mcp config set-routing on         # restore auto-routing
web-search-plus-mcp config set-priority you,serper,exa,firecrawl,tavily,linkup,parallel,brave,keenable
web-search-plus-mcp config set-extract-priority serper,tavily,exa,linkup,parallel,firecrawl,you,keenable
web-search-plus-mcp config set-fallback serper
web-search-plus-mcp config disable parallel
web-search-plus-mcp config enable parallel
web-search-plus-mcp config set-auto-allow parallel on
web-search-plus-mcp config set-threshold 0.45
web-search-plus-mcp config reset --yes

Use --config-path /path/to/config.json or WEB_SEARCH_PLUS_CONFIG=/path/to/config.json for isolated MCP host installs. Provider secrets stay in environment variables; routing behavior stays in config.json. Search and extraction priorities are independent. If an extraction priority lists only selected providers, the remaining extract-capable providers are appended in the public registry order.

Other presets:

• minimal — You.com only

• lean — You.com + Linkup

• starter — You.com + Serper + Linkup

• all — every supported provider env var

🤖 Grok Build plugin

This repository is also a Grok Build plugin. Grok reads .mcp.json (the same pinned uvx server as above), skills/web-search-plus/SKILL.md (when and how to use web_search and web_extract), and .grok-plugin/plugin.json (metadata).

Install directly from the repository:

grok plugin install robbyczgw-cla/web-search-plus-mcp

Or pick web-search-plus from the marketplace (/plugin inside Grok Build). uv must be installed; provider keys come from your environment, the plugin ships none. Check with grok mcp doctor.

⚙️ MCP host config

Canonical stdio snippet for Claude Desktop, Cursor, NanoBot, or Hermes native MCP:

{
  "mcpServers": {
    "web-search-plus": {
      "command": "uvx",
      "args": ["web-search-plus-mcp"],
      "env": {
        "YOU_API_KEY": "your_you_key",
        "SERPER_API_KEY": "your_serper_key",
        "LINKUP_API_KEY": "your_linkup_key"
      }
    }
  }
}

Common places to paste this snippet:

• Claude Desktop macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

• Claude Desktop Windows: %APPDATA%\\Claude\\claude_desktop_config.json

• Claude Desktop Linux: ~/.config/Claude/claude_desktop_config.json

• Cursor: project/user MCP config using the same mcpServers shape

• Hermes native MCP: ~/.hermes/config.yaml under mcp_servers with equivalent command/env fields

You can also place a .env file next to the package/project with the same variables.

🔌 MCP protocol compatibility

The server uses MCP Python SDK v2 and keeps the stdio boundary dual-era compatible:

• modern clients negotiate the stateless MCP 2026-07-28 protocol through server/discover

• legacy handshake clients continue to negotiate MCP 2025-11-25

The package exposes stdio only; the specification's Streamable HTTP session removal, subscriptions, OAuth changes, tasks, and multi-round-trip requests do not alter the two-tool Web Search Plus surface. Both protocol paths are covered by real subprocess tests.

🔎 Search Providers

• You.com — fast source-result provider for current and multilingual search

• Serper — Google-style facts, news, shopping, local queries

• Exa — semantic discovery, GitHub/docs, arXiv/academic, and OSS discovery

• Firecrawl — web search plus scrape-ready content

• Parallel — LLM-ready web search with long excerpts; joins automatic routing when a key is configured (PARALLEL_API_KEY)

• Tavily — research and analysis

• Linkup — source-backed grounding/citations

• Brave — explicit-only independent web index by default (BRAVE_API_KEY, auto_allow=false)

• SearXNG — privacy-first self-hosted meta-search

• SerpBase — explicit-only Google SERP API (SERPBASE_API_KEY, auto_allow=false)

• Querit — explicit-only multilingual, real-time AI search (QUERIT_API_KEY, auto_allow=false)

• Keenable — independent web index with search and extraction (KEENABLE_API_KEY, or opt-in keyless public tier; off by default)

• DonSeTch — explicit-only local Search/Fetch provider through a separately installed stdio executable (DONSETCH_BIN)

• Octen via Monid — explicit-only source-result web search with native recency and domain filters (MONID_API_KEY, auto_allow=false)

• TinyFish — explicit-only BYOK source-result search (TINYFISH_API_KEY, auto_allow=false)

Octen source search via Monid

The adapter executes Octen's /search endpoint through Monid's documented HTTP API for ranked links and highlights. It explicitly disables full-content retrieval and does not call Octen's answer or Broad Search APIs. Configure MONID_API_KEY from Monid, then select provider="octen"; automatic routing remains unchanged unless you deliberately enable auto_allow. Access and billing use Monid's prepaid wallet; see Monid for current pricing and terms.

TinyFish source search

Configure your own TINYFISH_API_KEY and select provider="tinyfish"; TinyFish remains outside automatic routing and fallback. Web Search Plus MCP does not provide, pool, proxy, or share TinyFish credentials. Review the TinyFish Search API reference and TinyFish Terms before use: the published Terms grant broad rights over Customer Data, including queries, for analysis, training, fine-tuning, evaluation, and model improvement. This integration therefore treats TinyFish as high risk and makes no no-training claim.

📄 Extract Providers

• Tavily — public default first choice; fastest reliable extraction in the v2.1 benchmark

• Exa — fast contents API, strong for docs/academic pages

• Linkup — clean markdown and source-grounded fetches

• Parallel — docs-focused fallback with full-content defaults of 60k characters per result / 120k total

• Firecrawl — robust scrape fallback, useful for JS-heavy/blocked pages

• You.com — LLM-ready snippets/content where available

• Keenable — keyed or explicitly opted-in public extraction

• Serper — fast webpage scraper extraction

• DonSeTch — explicit-only local Markdown fetch through a separately installed stdio executable

auto_routing.extract_provider_priority can override the auto-extraction order without changing search routing. Explicit provider calls still try the requested provider first.

Keenable keyless public access

Keenable exposes authenticated endpoints via KEENABLE_API_KEY. It also has keyless /public endpoints, but those are opt-in and disabled by default. Without a key, Keenable is treated as unconfigured unless you explicitly enable public egress:

{ "keenable": { "allow_public": true } }

or set:

KEENABLE_ALLOW_PUBLIC=1

Use an API key for private or production use. The public endpoint sends queries and fetched URLs to a shared unauthenticated service and remains near the tail of the public default fallback order unless the operator configures a different extraction priority.

DonSeTch local provider

DonSeTch 4.2.9 is an independent AGPL-3.0-only project. It is not bundled with this package. Web Search Plus starts the separately installed executable as a stdio MCP process using DONSETCH_BIN and projects its web_search and web_fetch results into the normal source-only envelopes.

DonSeTch remains explicit-only unless the operator enables auto_allow. The adapter was tested for stdio initialization, Search, Fetch, and structured errors; browser-based retrieval depends on the host environment and is not guaranteed.

See the DonSeTch setup and security guide for installation, configuration, licensing, migration, and tested boundaries.

Private/internal extraction target guard

web_extract blocks user-supplied target URLs that point at private or internal networks before any provider is called. This covers loopback, RFC1918, CGNAT/shared-address ranges, IPv6 local/mapped-private ranges, multicast, cloud metadata hosts, and hostnames resolving to private/internal IPs.

Operator-configured provider endpoints are separate: local Firecrawl-compatible backends can still run on 127.0.0.1 through provider config. If you intentionally need to extract trusted intranet URLs, opt in explicitly:

{ "extract": { "allow_private_urls": true } }

Leave this off for public/agent-controlled URL extraction.

GroktoCrawl / local Firecrawl-compatible backends

The Firecrawl provider can target a local Firecrawl-v2-compatible backend by overriding its search and scrape URLs in config.json. For example, a local GroktoCrawl instance listening on 127.0.0.1:8080 can be used without adding a separate provider:

{
  "firecrawl": {
    "api_url": "http://127.0.0.1:8080/v2/search",
    "scrape_url": "http://127.0.0.1:8080/v2/scrape"
  }
}

Keep FIRECRAWL_API_KEY configured if your backend enforces bearer authentication; local development instances may ignore the header. This does not make GroktoCrawl the default and does not claim coverage for every Firecrawl endpoint.

🛠 MCP Tool Reference

This MCP server exposes exactly two stable, source-only tools: web_search and web_extract. Use web_search for source discovery and let the MCP host synthesize from those sources when needed; the server itself does not generate answers or truth claims.

The Hermes plugin exposes the same stable capability as web_search_plus and web_extract_plus; the names differ because MCP and Hermes use different tool surfaces.

Response detail (both tools)

Both tools return one JSON TextContent. The new optional detail argument accepts "compact" or "full". Selection order is argument > WEB_SEARCH_PLUS_MCP_DETAIL environment variable > mcp.detail in config.json > "compact". Only these two values are accepted. The tool schema deliberately has no fixed default, so clients can omit the argument and inherit the server setting.

For clients that cannot pass detail, put WEB_SEARCH_PLUS_MCP_DETAIL=full in the server process environment or the MCP client's env block, or add this to the existing config file (selected by WEB_SEARCH_PLUS_CONFIG):

{"mcp": {"detail": "full"}}

full preserves the 5.1.2 response structure, including contract_version, observations, provider attempts, routing receipts, policy actions, dedup clusters, source diversity and requested quality reports. Clients that parse those fields must opt in to full. Routing, provider calls, caching and extraction limits are unchanged.

compact includes detail: "compact" and a hint for requesting the full contract. Search keeps titles, URLs, snippets, publication dates when available, and small source-type and fetch-priority labels. It also includes a short routing summary (failed or empty providers before the serving provider, excluding cancelled hedges), cache hits/age, short warning strings, and research providers merged. Research page counts are included only when extraction receipts provide a count; today's canonical search response may omit them. quality_report: true requires detail: "full" to return the detailed report. Failures keep status: "failed", a short error, provider reasons, setup instructions and environment-variable names; MCP isError remains true.

Compact extraction keeps one result per requested URL, in request order, with url, requested_url, title, bounded content, per-URL errors, requested spans and full_text storage references when present. A truncated page reports whether storing its full text succeeded; successful storage retains its reference, character count and checksum. Compact mode does not expose unbounded observation text.

web_search

Use for source discovery, current events, prices, weather, sports lineups, schedules, and whenever you want the raw search landscape first.

Parameters:

• detail — compact (server default) or full; overrides env/config

• query — required search query

• provider — auto, serper, serpbase, brave, tavily, querit, linkup, exa, firecrawl, parallel, you, searxng, keenable, donsetch, octen, tinyfish

• count — results to return, default 5, max 20

• depth — Exa depth: normal, deep, deep-reasoning

• time_range — hour, day, week, month, year

• freshness — unified day, week, month, or year recency request

• search_type — search or Serper-native news

• country / language — explicit locale overrides

• include_domains / exclude_domains — domain allow/deny lists

• mode — normal or research

• quality_report — include routing/result diagnostics with detail: "full"

• research_time_budget — best-effort wall-clock budget for research mode

• no_cache — bypass the search cache and fetch live results

• cache_ttl — search-cache TTL in seconds (default 3600)

Example MCP arguments:

{
  "query": "latest AI hardware news",
  "provider": "auto",
  "count": 5,
  "quality_report": true,
  "detail": "full"
}

web_extract

Parameters:

• detail — compact (server default) or full; overrides env/config

• urls — required list of URLs

• provider — auto, tavily, exa, linkup, parallel, firecrawl, you, keenable, serper, donsetch

• format — markdown or html

• include_images — include image metadata when supported

• include_raw_html — include raw HTML when supported

• render_js — render JavaScript before extraction when supported

• spans — select deterministic semantic spans from extracted text

• spans_query — optional query used to rank semantic spans

Example MCP arguments:

{
  "urls": ["https://example.com"],
  "provider": "auto",
  "format": "markdown"
}

Result shape: results holds one entry per requested URL, in request order. Each entry has url (where the page was actually fetched, after redirects and normalization), requested_url (the URL you passed in, so you can match results to inputs without relying on order), content, and error when the URL failed. URLs past the per-call limit (default 10) and URLs a provider never reached after a rate-limit, credit or auth error come back as their own entries with error: "not tried: …"; pages fetched before such an error are kept. requested_url is additive: url is unchanged.

🧠 Classic Routing v2 examples

• 東京 AI ニュース 今日 → You.com multilingual/current search

• arXiv 2024 LLM scaling laws → Exa academic discovery

• CVE-2025 openssl advisory → Brave security/current search

• best bookshelf speakers under 1000 EUR Austria → Brave shopping/local search, with Serper as a later fallback

• open source alternatives to Linear → Exa/Firecrawl OSS discovery

• recent RAG vs fine-tuning benchmark sources → source-result discovery; the MCP host may synthesize from returned sources

Guarded providers can still be called explicitly. To let one participate in provider="auto", opt in:

web-search-plus-mcp config set-auto-allow parallel on
web-search-plus-mcp config set-auto-allow parallel off
web-search-plus-mcp config set-auto-allow donsetch on
web-search-plus-mcp config set-auto-allow donsetch off

Development

See CONTRIBUTING.md for project boundaries, portable-engine sync rules, provider requirements, security constraints, and pull-request expectations.

Run the same gates as CI:

python -m pip install --upgrade pip build
python -m pip install -e ".[test]"
python -m pytest tests/ -q -p no:cacheprovider
ruff check --config pyproject.toml .
python -m compileall -q web_search_plus_mcp tests
python scripts/gen_contract_v3_schemas.py --check
python -m build

The GitHub Actions workflow runs the test suite on Python 3.10, 3.11, and 3.12, then verifies Ruff, byte-compilation, wheel creation, source-distribution creation, and wheel/sdist parity.

Credits

Thanks to LugMuad for the native Hermes integration proposal and testing in #125. That opt-in adapter ships in the Hermes plugin, not this MCP package.

Built on the Web Search Plus routing engine and packaged as a standalone MCP server.

License

MIT © 2026 robbyczgw-cla