web-mcp-server

MCP server that exposes web_search, web_fetch, and research tools, allowing LLM applications to search the web, fetch page content, and research ArXiv papers.

Tools

web_search

Search the web via DuckDuckGo.

Returns a list of {title, url, description}.

web_fetch

Fetch a URL and return cleaned markdown content.

Returns {content, title, url, content_type}.

research

Search ArXiv papers, fetch HTML content, chunk, and return the most relevant chunks ranked by BM25.

Returns {query, sources, chunks, total_chunks_analyzed}.

Related MCP server: LLM Researcher

Deep Research Prompt

The server exposes a deep_research prompt via the MCP protocol. Clients can discover and call it with a query parameter to get a system prompt for iterative deep research.

Usage: Call deep_research(query="...") to get a prompt that guides an LLM agent to orchestrate web_search, web_fetch, and research in multi-round investigation following the CoSearch deep search pattern.

Quickstart

# Install dependencies
uv sync

# Run the server (SSE on port 8642, default)
uv run python -m src.server

# Run with streamable HTTP transport
TRANSPORT=streamable-http uv run python -m src.server

Environment Variables

Health Check

A health check endpoint is available at the root path:

curl http://localhost:8642/
# {"status": "ok"}

Connect via Claude Code

# SSE (default)
claude mcp add web-tools http://localhost:8642/sse

# Streamable HTTP
claude mcp add --transport http web-tools http://localhost:8642/mcp

Connect via MCP Inspector

npx -y @modelcontextprotocol/inspector
# Connect to http://localhost:8642/sse (SSE) or http://localhost:8642/mcp (streamable HTTP) in the UI

Project Structure

src/
├── server.py            # MCP server setup + tool/prompt registration
├── tools/
│   ├── web_search.py    # DuckDuckGo search logic
│   ├── web_fetch.py     # URL fetch, HTML cleaning, markdown conversion
│   └── research.py      # ArXiv paper search, fetch, chunk, BM25 ranking
└── utils/
    └── html.py          # HTML title extraction

Requirements

• Python >= 3.10

• uv (recommended) or pip

Dependencies