keyless-web-search-mcp

Standalone MCP stdio server: the keyless web_search tool over a self-operated engine pool (Bing, 360, Baidu, Google, Naver, Yandex, DuckDuckGo) and — opt-in via ENABLE_WEB_FETCH=1 — web_fetch (anonymous HTTP(S) page reader). Each search keeps at most two usable engines, routes default priorities by query language, back-fills failed or irrelevant engines within a bounded attempt budget, relevance-ranks results, merges them, and dedupes by canonical URL. Self-contained directory, zero build step, not part of the harness package system — move it anywhere.

Why this exists

The harness's built-in web_search (DeepSeek provider) needs DEEPSEEK_API_KEY and sends every query to the DeepSeek cloud. This server is the local-model-friendly alternative: no keys, no vendor search API, works on mainland-China and HK networks (Bing/360/Baidu from mainland lines, Bing/360/Naver/Yandex from the tested HK line; DuckDuckGo is not reachable from any of them).

web_fetch exists for the same reason on the read side: it was built when the harness web profile mounted a search provider but no fetch provider, so the built-in web_fetch tool failed every call with WEB_PROVIDER_UNAVAILABLE. That gap is now filled by the harness, so web_fetch registers only when ENABLE_WEB_FETCH=1 is set in the server's env — the default mount is search-only, and a session that already has a harness fetch tool does not see two readers.

Versioning. The version in package.json / SERVER_VERSION is bumped only by the project owner when cutting a release; working-tree changes never bump it.

Related MCP server: WET - Web Extended Toolkit

Run

npm install --cache ./.npm-cache   # deps: @modelcontextprotocol/sdk, zod
node index.js                      # speaks MCP over stdio (web_search only)
ENABLE_WEB_FETCH=1 node index.js     # also register the web_fetch reader
npm test                            # run deterministic ranking/fallback tests

Quick probe without a client:

printf '%s\n' \
  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0.0.1"}}}' \
  '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  | node index.js

Install in Claude Code

Install it for the current user so every Claude Code project can use it. Resolve both paths when registering the server; this avoids depending on the shell's working directory or PATH later:

claude mcp add --transport stdio --scope user web-search-self -- "$(command -v node)" "/absolute/path/to/keyless-web-search-mcp/index.js"
claude mcp get web-search-self

Use --scope local instead to keep it private to the current project, or --scope project to write a shareable .mcp.json. Claude Code stores user/local registrations in ~/.claude.json; MCP server definitions do not belong in settings.json.

After registration, claude mcp list should report web-search-self as connected. The tool appears as mcp__web-search-self__web_search (and mcp__web-search-self__web_fetch too, when the registration's env sets ENABLE_WEB_FETCH=1).

Tools

web_search

web_search({ query, count?, maxSources?, engines? })

Two-source policy. The pool is routed by query language when engines is omitted: Chinese starts with 360/Baidu/Bing; English starts with Bing/Google/Naver/360 so the four-attempt budget retains working fallbacks on the tested mainland/HK networks. An explicit engines list remains authoritative. Candidates run in bounded parallel rounds; an engine consumes a source slot only after it returns parseable results that pass conservative lexical relevance filtering. Failed, challenged, empty, or irrelevant engines are back-filled up to the attempt and collector budgets, and diagnostics are shown in an "Engine note" line. Results are ranked by query overlap with a small source-quality adjustment: obvious repost/content-farm signals are penalized, while official/docs/education/GitHub signals are favored without hard-blocking ordinary sites. Merging remains round-robin interleaved (engine A's 1st, engine B's 1st, A's 2nd, …), and canonical URL dedupe removes fragments, common tracking parameters, and safe www. differences. Resilience: an engine that throws a bot-challenge diagnostic enters a 30-minute challenge memory and is skipped — without consuming an attempt — on subsequent searches; if the relevance filter discards every engine's results, the search returns the parsing engines' raw top results with an explicit note instead of failing empty; long queries relax the relevance threshold proportionally (the 0.3 bar holds for up to 4 terms), and boolean words like or no longer count as query terms.

Result: numbered list of title [engine] / real destination URL / snippet. Link cleanup per engine: Bing click-tracker links decode locally (base64 u parameter); 360 reads the data-mdurl attribute; Baidu reads the block's mu attribute (direct URL, with one best-effort redirect GET only for legacy link?url= wrappers); Naver and Yandex titles carry the direct URL in the anchor; Google /url?q= wrappers unwrap; DuckDuckGo /l/?uddg= trackers unwrap. Per-engine probe budgets: 10 s (Bing, Baidu), 8 s (360, Naver, Yandex), 5 s (Google, DuckDuckGo) — all run in parallel, so the round is only as slow as its slowest member.

web_fetch (opt-in: ENABLE_WEB_FETCH=1)

web_fetch({ url, maxChars? })

Anonymous public-web read, no credentials: browser User-Agent, at most five redirects, 20 s wall clock covering headers and body, response body cut at exactly 5 MB. Every initial and redirect target is checked after DNS resolution; loopback, private, link-local, carrier-grade NAT, reserved, multicast, and local hostnames are rejected to prevent SSRF into the machine or cloud metadata services. HTML is stripped to visible text (script/style/noscript/svg/head/iframe/canvas/form dropped — an unclosed script/style/noscript/iframe cuts the document there, matching HTML5 raw-text parsing — block boundaries become newlines, the <title> is carried in the header), and up to 30 unique in-page links are appended (relative refs resolved against the final URL, fragments and non-http schemes skipped, low-signal search/social hosts dropped) so the model can follow up without re-searching. text/* and XML media pass through entity-decoded using their declared charset; JSON, JavaScript, and form bodies carry no entity layer and pass through byte-exact; binary media types are rejected. A status-header note flags large pages that yield little visible text (likely JavaScript-rendered). Output is a status header — status, final url (after redirects), content-type, truncation — followed by the content. Non-2xx answers return the body head plus a note (403/429 read as bot-check or paywall), never a fabricated page.

Engines and their status

Verified 2026-07 from three lines (an earlier mainland line, a Shanghai Telecom line, and a Hong Kong Zenlayer datacenter line):

Deliberately not in the pool: Sogou — 302s to sogou.com/antispider/, the same IP-wall class Baidu covers. Also scanned and rejected from the HK line: Mojeek (serves a captcha page), Ecosia (403 "Ecosia Firewall"), MetaGer (redirects to a non-result page), and Yahoo/Brave/Qwant/Startpage/goo.ne.jp (TCP-unreachable from both mainland and HK lines).

Google wall: what was tried

The 200 response from this IP is not a block but a JS challenge interstitial (~90 KB of obfuscated/encrypted JavaScript; the no-JS path is a meta-refresh into a dead end). The wall turns out to be two layers:

1. JS challenge (computational) — decoded, it computes a proof value and sets an SG_SS cookie (5-minute expiry), then reloads. This layer is solvable outside a browser: running the page's scripts in plain Node.js with a light DOM shim (cookie jar, navigator, Image, document) completed the computation, and the resulting SG_SS cookie was accepted once — Google answered 200 and issued the NID/AEC trust cookies it otherwise never sends to this IP.

2. Session-pattern layer (behavioral) — subsequent plain-HTTP follow-ups, including an exact replica of the script's own reload flow (emsg=SG_REL + matching sei, jar-merged cookies, browser headers), escalate to the google.com/sorry 429 anomaly wall. This layer judges the whole session (TLS/HTTP2 fingerprint, request pacing, method mix), which Node's OpenSSL/undici stack does not match.

Client-side levers that were tested and did not change layer 1's answer: the gbv=1 parameter, cookie warm-up, full browser fingerprint headers, the retry/enablejs flow, the /m entry, text-browser/feature-phone/IE6/old-Android UAs, a spoofed Googlebot UA (Google validates it), CONSENT cookies, POST submission (405), alternate TLDs, and the "click here" emsg=SG_REL link without the cookie. Google-proxy alternatives were also unreachable or walled from this network: Startpage and Qwant time out, Mojeek serves a captcha page, Ecosia 403s.

Conclusion: layer 1 is Node-solvable (demonstrated); layer 2 is not, from a non-browser network stack. The only reliable paths to Google from this machine are a clean-IP proxy or a real browser context (headless Chromium); both sit in the same IP/traffic-pattern layer that a scraper cannot honestly out-argue, and repeated probing risks extending the IP's anomaly window — so this server does not attempt them. Google stays a back-fill engine that activates on networks where it will serve a plain-HTML SERP.

Mount into DeepSeek Harness

Add to your cordis.yml (the MCP bridge hot-reloads this entry):

- id: mcp-search
  name: '@deepseek-ai/dsh-mcp-client'
  config:
    serverName: search
    transport: stdio
    command: node
    args: ['/absolute/path/to/keyless-web-search-mcp/index.js']
    # env: { ENABLE_WEB_FETCH: '1' }   # opt-in: also register web_fetch

The model then sees the tool as mcp__search__web_search; with the commented env line enabled it also sees mcp__search__web_fetch.

Limitations

• Search quality is heuristic: lexical relevance handles obvious off-topic results and Chinese n-grams; internal search-engine pages are excluded; content-farm/repost signals only adjust ordering, not a universal trust judgment. A relevant result is not proof that its claims are correct — fetch and cross-check important facts.

• Scraping, not an API: layout changes can break a parser; the failure is loud ("no parseable organic results"), never fabricated. The Google and DuckDuckGo parsers are written from documented SERP structure and are not live-verifiable from this network — tune them the first time their engines actually answer.

• Rate control: both public engines tolerate casual use; hammering draws bot challenges.

• Query leaves the machine for the consulted engines (that is the price of keyless search); with the two-source cap, at most two of them see a query per search.

• Fetch is a keyless public-page reader, not a browser or intranet client: JS-rendered content is invisible to it (same class of limit as the search parsers); pages that 403/429 anonymous clients are reported, not bypassed. Local/private/reserved network targets are deliberately blocked, including redirect destinations.