web-browser-mcp

Query-driven web search + page fetch for AI agents. The agent gives it a natural-language question, gets back ranked results with snippets (and optionally full page content) ready to cite.

A small MCP server with two tools that compose cleanly:

Multi-engine fallback: Bing via headless Chromium → DuckDuckGo HTML via httpx. No API keys required.

This is the shape you want when the agent is answering a question like "what's the latest model context protocol spec?" or "find a good tutorial on asyncio". The agent doesn't need to know which engine to use, what URL to fetch, or how to render the SERP — it just calls web_search and gets back ready-to-inject context.

Install

git clone https://github.com/ptrken01/web-browser-mcp
cd web-browser-mcp
uv venv .venv --python 3.11
uv pip install --python .venv/bin/python -e ".[dev]"
.venv/bin/playwright install chromium

(If you have an existing venv from elsewhere, just uv pip install -e . in it and run playwright install chromium once. The .[dev] extra adds pytest, mypy, and ruff.)

Related MCP server: Search Proxy MCP

Run

stdio (Claude Desktop, Cursor)

.venv/bin/web-browser-mcp

In your client's MCP config:

{
  "mcpServers": {
    "web-search": {
      "command": "/absolute/path/to/web-browser-mcp/.venv/bin/web-browser-mcp"
    }
  }
}

streamable-http (llama-ui, Open WebUI, browser clients)

.venv/bin/web-browser-mcp --transport streamable-http
# Default: http://127.0.0.1:8766/mcp

In llama-ui's MCP server settings, add an HTTP transport pointing at http://127.0.0.1:8766/mcp. CORS is open for localhost / 127.0.0.1 by default.

Tools

web_search(query, limit=5, include_content=False, ...)

Give it a natural-language question or topic. Returns ranked results with title, URL, and snippet, ready to be cited in your answer.

{
  "name": "web_search",
  "arguments": {
    "query": "model context protocol specification",
    "limit": 5,
    "include_content": false
  }
}

Parameters:

Response:

{
  "query": "model context protocol specification",
  "engine": "duckduckgo",
  "count": 5,
  "results": [
    {
      "title": "Official site",
      "url": "https://modelcontextprotocol.io",
      "snippet": "Model Context Protocol",
      "engine": "duckduckgo"
    },
    {
      "title": "What is the Model Context Protocol (MCP)?",
      "url": "https://modelcontextprotocol.io/docs/getting-started/intro",
      "snippet": "MCP (Model Context Protocol) is an open-source standard for connecting AI applications to external systems...",
      "engine": "duckduckgo"
    }
  ],
  "duration_s": 1.234
}

When include_content=True, each result additionally has:

{
  "content": "Main text extracted from the page...",
  "content_chars": 3421
}

get_page_content(url, max_chars=10000)

For the "I have a specific URL and want its content" case.

{
  "name": "get_page_content",
  "arguments": {
    "url": "https://example.com/article",
    "max_chars": 5000
  }
}

Response:

{
  "url": "https://example.com/article",
  "final_url": "https://example.com/article",
  "title": "Example Article",
  "text": "Main content of the page...",
  "text_chars": 3421,
  "duration_s": 0.5
}

Engines and fallback

The tool tries engines in engine_order. The first one to return ≥1 result wins. If all engines fail, the response has an error field with a stable string code.

In practice, DDG HTML is the workhorse — it's the engine that succeeds most often in test runs. Bing is the upgrade path for richer SERP data when the agent has a fresh IP and the captcha doesn't trip.

Error handling

All errors come back as structured error fields, not exceptions:

Configuration (env vars)

Test

.venv/bin/python -m pytest

14 tests pass. The end-to-end test (tests/test_server_e2e.py) boots real uvicorn + real FastMCP + a real Playwright browser, performs the MCP initialize handshake, and calls web_search against real search engines. This is the test that catches lifespan-bug classes from searxng-mcp-scraper — unit tests that call tool functions directly with hand-built state would miss them.

How it fits with the rest of the toolkit

Use web_search first for general questions. Drop to get_page_content or searxng-mcp-scraper.fetch when you have a URL. Use scrape_blog for "read this whole blog" use cases.

License

MIT

Live site

All products + free articles: autoincomesys.com — local-first AI dev notes, build-once digital products (Gumroad).

Related

• Local-First AI Dev Notes (SEO site)

• web-browser-mcp

• fastapi-doc-gen