web-research-mcp

An MCP server that exposes web search and page reading as standard tools, resources, and a prompt template — so any MCP client (Claude Desktop, Claude Code, or another agent) can research a topic without each one reimplementing search/fetch/extract from scratch.

Why this exists

I'd already built factcheck-agent, a LangGraph agent with its own search-fetch-extract loop hardcoded into its graph. That logic is genuinely reusable — any agent that needs to look something up on the web needs roughly the same thing. This project pulls that capability out into a standard MCP server instead of leaving it locked inside one agent's internals.

Related MCP server: groundwork

What it exposes

Tools

• search(query, max_results) — title/url/snippet results from DuckDuckGo (or Tavily, if configured).

• fetch(url) — fetches a page and extracts its readable text. Returns an error field instead of raising on failure (PDFs, JS-rendered pages, and blocked requests are all reported, not silently swallowed).

• research(topic, depth) — search, then fetch the top results in one call, returning a source list with excerpts. Also appends everything gathered to the current session.

Resources

• research://session — everything gathered by research calls so far, readable without re-searching.

Prompts

• fact_check(claim) — a template that frames a claim as a research task and requires citations for any verdict.

Install

git clone https://github.com/orackle/web-research-mcp.git
cd web-research-mcp
python -m venv .venv
.venv\Scripts\activate   # or source .venv/bin/activate on macOS/Linux
pip install -r requirements.txt

Run

Local inspector (for poking at tools/resources by hand):

mcp dev web_research_mcp/server.py

Register with Claude Desktop:

mcp install web_research_mcp/server.py

Configuration

All optional — defaults work with no API keys, using DuckDuckGo:

Stack

• MCP Python SDK (mcp 2.x — MCPServer, formerly FastMCP)

• httpx + BeautifulSoup for fetching/extraction

• DuckDuckGo (ddgs) / Tavily for search

• Pydantic for typed tool schemas

Eval

eval/run_eval.py exercises all three tools against the real network (live search engines, real pages) — not mocked — and checks each result against what that case expects, including two deliberate failure cases (an unreachable domain, an image URL) to confirm errors surface as data instead of crashing the tool.

python -m eval.run_eval

Latest run, 9/9 passed:

[1/9] PASS search   (1.35s)  5 results
[2/9] PASS search   (1.65s)  3 results
[3/9] PASS search   (1.18s)  3 results (any count ok)
[4/9] PASS fetch    (1.10s)  text=4000 chars, error=None
[5/9] PASS fetch    (0.52s)  text=2288 chars, error=None
[6/9] PASS fetch    (0.54s)  error='fetch failed: [Errno 11001] getaddrinfo failed'
[7/9] PASS fetch    (0.99s)  error='unsupported content-type: image/jpeg'
[8/9] PASS research (4.24s)  2 sources, 2 fetched
[9/9] PASS research (3.01s)  3 sources, 2 fetched

9/9 cases passed

Per-tool metrics:
  fetch      4/4 passed  latency: mean=0.79s median=0.77s max=1.10s
  research   2/2 passed  latency: mean=3.62s median=3.62s max=4.24s
  search     3/3 passed  latency: mean=1.39s median=1.35s max=1.65s

Full output: eval/eval_results.log.

Verified against the MCP Inspector

The server connects cleanly to the official MCP Inspector over stdio, and the research tool was run live from it end to end (real Wikipedia fetch, 5.5s round trip):

The research://session resource correctly reflects the accumulated state from that call:

Known limits

• The session resource is a single in-memory session per server process — fine for one local client at a time, not built for multi-tenant use.

• fetch only handles HTML pages; PDFs and JS-rendered pages come back as an error, not a partial/garbled result.

• DuckDuckGo has no API key and is occasionally rate-limited or flaky — that's the tradeoff for zero-setup search; swap to Tavily via env vars if you need more reliability.