Skip to main content
Glama
mcp-webfetch

mcp-webfetch

by Mahgal

mcp-webfetch

Give your AI agent the web — as clean Markdown, not raw HTML. A tiny Model Context Protocol server that fetches a page and returns readable Markdown plus its link list, in ~600 lines of Python with zero required dependencies.

Python License: MIT MCP Dependencies Tests

Why

Agents fed raw HTML waste context on tags, scripts and navigation chrome. mcp-webfetch does one job well: turn a URL into the part a model actually wants to read — title, headings, paragraphs, emphasis, code and links — and hand back the page's links when the agent wants to crawl onward.

Related MCP server: read-website-fast

Features

  • fetch_url(url) — fetch a page and return clean Markdown (title as #, headings, lists, blockquotes, code blocks, [text](href) links with relative URLs resolved).

  • extract_links(url) — return the page's links as a JSON array: de-duplicated, fragments stripped, only http/https kept.

  • Hard limits built in — 10 s timeout, 1 MB response cap, non-HTML content types rejected with a clear error. No runaway fetches burning agent context.

  • Zero required dependencies — fetching and HTML→Markdown conversion are standard-library only (urllib + html.parser). No readability/BeautifulSoup/html2text.

  • Official SDK when available, honest fallback when not — if the mcp Python SDK is installed it serves the tools (both the v1 FastMCP and v2 MCPServer APIs). Otherwise a built-in minimal JSON-RPC-over-stdio implementation speaks the same MCP subset (initialize, tools/list, tools/call, ping). Set MCP_WEBFETCH_FORCE_STDIO=1 to force the fallback.

  • Tested — 20 pytest cases against a local HTTP server and fixtures: conversion, link extraction, redirects, timeout, size limit, non-HTML rejection, and a full stdio protocol round-trip.

Quick start

git clone https://github.com/Mahgal/mcp-webfetch.git && cd mcp-webfetch
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[sdk]"        # or plain `pip install -e .` — stdlib fallback, no SDK needed
python -m mcp_webfetch         # server runs on stdio; it waits for an MCP client

Run the local demo (serves a page on localhost, calls both tools, prints real results — no internet needed):

pip install -e ".[dev]"
python examples/demo.py
pytest

Use it in an MCP client

Claude Desktop

Add to claude_desktop_config.json (macOS: ~/Library/Application Support/Claude/, Windows: %APPDATA%\Claude\):

{
  "mcpServers": {
    "webfetch": {
      "command": "/absolute/path/to/mcp-webfetch/.venv/bin/python",
      "args": ["-m", "mcp_webfetch"],
      "cwd": "/absolute/path/to/mcp-webfetch"
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project (or the global MCP settings):

{
  "mcpServers": {
    "webfetch": {
      "command": "/absolute/path/to/mcp-webfetch/.venv/bin/python",
      "args": ["-m", "mcp_webfetch"],
      "cwd": "/absolute/path/to/mcp-webfetch"
    }
  }
}

Restart the client, then ask your agent things like "Fetch https://example.com and summarize it" or "List the links on this page" — it will call fetch_url / extract_links.

Real demo output

Verbatim output of python examples/demo.py (run 2026-11-09, Python 3.12, mcp SDK 2.3.0; the localhost port is ephemeral and differs per run):

Serving demo page at http://127.0.0.1:40503/sample_page.html

(using the official MCP Python SDK client)

Tools: ['fetch_url', 'extract_links']

--- fetch_url(http://127.0.0.1:40503/sample_page.html) ---
# mcp-webfetch Demo Page

This page is served locally by `examples/demo.py` so the demo needs **no internet access** and touches no live website.

## What the tools do

- `fetch_url` turns this HTML into clean Markdown
- `extract_links` lists the links below, de-duplicated

Read the [MCP specification](https://modelcontextprotocol.io), browse [example.com](https://example.com), or revisit [the spec again](https://modelcontextprotocol.io) (duplicate on purpose).


--- extract_links(http://127.0.0.1:40503/sample_page.html) ---
[
  "https://modelcontextprotocol.io",
  "https://example.com"
]

Without the SDK installed, the same demo runs through the built-in JSON-RPC stdio client and prints the same tool results.

Project layout

mcp_webfetch/
  converter.py     HTML -> Markdown + link extraction (html.parser only)
  fetcher.py       urllib fetching with timeout / size / content-type limits
  tools.py         fetch_url + extract_links, transport-independent
  server.py        entry point: official SDK if present, fallback otherwise
  stdio_server.py  minimal MCP JSON-RPC-over-stdio implementation (stdlib)
tests/             pytest suite (fixtures + local HTTP server, no network)
examples/          demo.py + sample_page.html — local end-to-end demo

Design notes & limits

  • Conversion is intentionally simple: semantic HTML in, Markdown out. It does not run JavaScript, so JS-rendered pages return whatever the server-sent HTML contains.

  • Tables are flattened to text lines rather than Markdown tables; images keep ![alt](src) form.

  • Only http:// and https:// URLs are accepted; file:, ftp: etc. are refused.

  • Errors (timeout, HTTP status, non-HTML, oversized body) come back to the agent as MCP tool errors with a human-readable message.

License

MIT — see LICENSE.

Available Tools

2 tools
fetch_urlA

Fetch a web page and return its content as clean Markdown (title, headings, paragraphs and links preserved). Times out after 10s and rejects non-HTML or oversized responses.

Inputs: url (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.8/5.0

    Scored across 2 tools

    Disambiguation4/5

    The two tools differ mainly in output shape: fetch_url returns Markdown while extract_links returns a JSON link array. There is mild overlap because fetch_url explicitly preserves links, so an agent could plausibly get links via either tool, but the descriptions make the intended split reasonably clear.

    Naming Consistency5/5

    Both names follow a strict verb_noun snake_case pattern (fetch_url, extract_links) with no stylistic deviation. The convention is predictable and immediately readable.

    Tool Count3/5

    Two tools is thin for a general-purpose web-fetch server; a single narrow utility would be fine, but this surface leaves little room for variation in fetch behavior. It is not bloated, just minimal.

    Completeness3/5

    The core fetch-and-parse loop is covered, but notable gaps exist: no raw HTML/JSON fetch, no custom headers or method options, and no main-content/text-only extraction. Agents hitting non-HTML endpoints or auth-gated pages have no path forward.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents and LLM pipelines to fetch and crawl web pages, extract readable content, and convert it into clean Markdown with preserved links while minimizing token usage.
      MIT
    • A
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents to fetch and crawl web pages, strip noise, and convert content into clean Markdown with preserved links for token-efficient scraping and downstream processing.
      MIT