JedSearch MCP Server

MCP server that gives agents a web search tool and a web reader, with two search engines fused per query and a tiered page reader. Plain streamable-HTTP MCP, API-key protected.

Tools

Split by cost on purpose: an agent that only needs a fact pays ~5 KB, not the 17–64 KB per call that the previous single web_search tool always returned.

Related MCP server: MCP MyZ Search

Search legs

Legs run concurrently and are merged with reciprocal-rank fusion (1/(60+rank), tie-break prefers Yandex). A result with "agree": 2 was ranked by both engines — the strongest signal available. Each leg reports {ok, ms, n} and, for SearXNG, engines_failed, so a degraded leg is visible to the caller instead of silently shrinking the result set.

Two Yandex API quirks are handled in yandex_client.py: FORMAT_JSON is rejected (only FORMAT_XML works, base64 in rawData), and sites[], pages[], maxpassages, window/timeInterval are accepted but silently ignored — hence site: in the query text and post-filtering by modtime for freshness.

Reader tiers

Per URL, cheapest first, stopping as soon as the text is usable:

1. httpx with SSRF validation on every redirect hop + trafilatura extraction (markdown, keeps the page date).

2. The Yandex cached copy of that page (yandexwebcache.net), learned from the search leg's saved-copy-url — gets past login walls, bot checks and geo-blocked origins.

3. Playwright sidecar (ws://127.0.0.1:3000) for JS-only sites. Images, fonts, media and stylesheets are aborted; service workers blocked.

render: "always" | "never" forces tier 3 on or off. Every page reports reliable plus the reason it failed, so an unreadable page is not mistaken for a missing fact. Pages are cached (TTL 1 h), searches cached for 10 min.

SSRF

net_guard.py validates scheme, host and every resolved IP before any request, for both the HTTP client and the browser's initial navigation, and re-validates each redirect target. Blocked: RFC1918/loopback/link-local (including 169.254.169.254), CGNAT, IPv6 private ranges, IPv4-mapped v6, and single-label hostnames — which is what stops the reader from reaching a container service name (http://<service>:9000) from inside the docker network. SSRF_ALLOW_PRIVATE=true or SSRF_ALLOWED_HOSTS= relax it deliberately.

Layout

config.py         every environment knob in one place
models.py         tool argument models (also the JSON schema agents see)
server.py         MCP wiring: three tools, auth middleware, logging
yandex_client.py  Yandex v2 XML client
searxng_client.py SearXNG JSON client
fusion.py         URL normalisation, dedupe, reciprocal-rank fusion, budgets
reader.py         tiered page reader (httpx → Yandex copy → Playwright)
net_guard.py      SSRF validation + redirect-safe GET
page_cache.py     TTL/LRU caches + counters
entrypoint.sh     Playwright sidecar + server

Configuration

See .env.example. Everything is read in config.py; nothing is hardcoded. SEARXNG_SECRET is for SearXNG itself (openssl rand -hex 32), not for this server.

Running

Both legs — compose.yaml starts SearXNG and jedsearch on one private network:

cp .env.example .env    # set YANDEX_SEARCH_API_KEY, JEDSEARCH_API_KEY, SEARXNG_SECRET
docker compose up -d    # jedsearch on :3001, SearXNG internal (no published port)

Single container, Yandex leg only:

docker build -t jedsearch .
docker run -d --name jedsearch --env-file .env -p 3001:3001 jedsearch

SEARXNG_URL empty disables the second leg, and backend: "yandex" skips it per call. searxng/settings.yml is the engine list that was measured on one network — re-measure on yours: GET /config lists every engine, GET /search?q=x&format=json&engines=<name> shows whether one actually answers from your egress. The container runs as uid 977 and chowns /etc/searxng at startup, so expect that directory to come back owned by it.

Smoke test — 401 without the key, 200 with it:

curl -s -o /dev/null -w '%{http_code}\n' -X POST localhost:3001/mcp \
  -H 'Content-Type: application/json' -H 'Accept: application/json, text/event-stream' \
  -H "x-api-key: $JEDSEARCH_API_KEY" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"c","version":"1"}}}'

The container runs two processes: the Playwright browser server (port 3000, internal) and the MCP server (port 3001, /mcp). entrypoint.sh waits for the browser and, if it never comes up, logs a warning and starts anyway — tier 3 is lost, the rest keeps working. Healthcheck is a TCP probe on 3001.

mcp is pinned <2 because the agents' MCP client (and this code) target the 1.x FastMCP API; SDK 2.x renamed it to MCPServer.