raw-web-search

An experimental MCP search server for Claude Code. Returns raw, verbatim web content instead of AI-rewritten summaries.

This is a research project, not a production tool. It explores whether raw web extraction can replace Claude Code's built-in WebSearch + WebFetch. The answer: it depends on the use case. See Eval Results for honest benchmarks.

What it does

One MCP tool call that searches the web, fetches pages, extracts content, and returns the most query-relevant excerpts — all verbatim from the source.

search("how to implement rate limiting in FastAPI")
→ 5 results with verbatim excerpts from real pages (~1,400 tokens total)

Also includes extract(urls) for direct URL content extraction (replaces WebFetch).

When to use this instead of WebSearch

Bottom line: Use raw-web-search when you need verbatim source text (code, docs, exact quotes) or want frictionless auto-approved searching. Use WebSearch when you need synthesized overviews or general knowledge.

Eval Results

Quantitative (20 queries)

Quality (5 queries, 25 ground-truth facts)

WebSearch wins on factual coverage because it uses an LLM (Haiku) to synthesize across 10+ full pages. We use embeddings to select the most relevant paragraphs from 5 pages — a fundamentally different (and less thorough) approach.

What we learned

1. AI synthesis > embedding selection for factual coverage. An LLM reading full pages and picking key facts will always beat cosine-similarity paragraph selection.

2. Verbatim text > AI summaries for code. Paraphrased code examples and API signatures are useless. Raw extraction preserves them.

3. Auto-approval is the killer feature. The biggest real-world advantage isn't content quality — it's eliminating click-per-search friction.

4. WebFetch is genuinely bad. 50% failure rate on real-world URLs. Our extract tool is a clear improvement.

How it works

Query
  → SearXNG (search 70+ engines via Docker)
  → Fetch HTML (httpx concurrent, 4s timeout)
  → Playwright fallback for failed URLs (JS-rendered pages)
  → Extract content (trafilatura, F1=0.958)
  → Split into paragraphs
  → Embed query + paragraphs (fastembed, all-MiniLM-L6-v2)
  → Select top chunks by cosine similarity (~1500 chars/result)
  → BM25 rank results
  → Return structured text

Prerequisites

• Python 3.10+

• Docker (for SearXNG search backend)

How Docker is used

SearXNG runs as a Docker container alongside Redis. The MCP server auto-starts containers on demand.

• Auto-start: Checks if SearXNG is reachable, runs docker compose up -d if not

• Port: localhost:8888 (configurable via SEARXNG_URL)

• Stopping: docker compose down in the project directory

Install

# Install globally (available in all projects)
claude mcp add -s user open-search -- uvx --from git+https://github.com/andresfortunato/raw-web-search.git open-search-mcp

# Set as default search (one-time — tells Claude to prefer this over WebSearch)
uvx --from git+https://github.com/andresfortunato/raw-web-search.git open-search-mcp --setup

# Optional: install Chromium for ~100% extraction success (vs 76% without)
playwright install chromium

Tools

search

Search the web and return extracted content.

search(query, max_results=5, include_domains=None, exclude_domains=None, time_range=None)

extract

Extract content from specific URLs. Direct replacement for WebFetch.

extract(urls, query=None)

Configuration

Architecture

src/open_search_mcp/
  server.py     # MCP server, tool definitions, Docker lifecycle
  searcher.py   # SearXNG client + BM25 scoring
  extractor.py  # URL fetching + trafilatura + Playwright fallback
  chunker.py    # Embeddings-based chunk selection (fastembed)
  cache.py      # TTL-based URL cache

Research

The research/ directory contains all eval data, benchmarks, and analysis:

• eval_a_v2.md — Factual coverage scoring (72% vs WebSearch's 96%)

• quality_eval.md — Direct content comparison (Eval C)

• baseline_comparison.md — Full baseline eval write-up

• eval_20_results.json — 20-query benchmark data

License

MIT