web-to-markdown-mcp
This server fetches web pages and converts their main content to clean Markdown, using a three-tier strategy to handle static pages, JavaScript-heavy sites, and bot-protected pages.
Fetch any URL as Markdown — retrieves the main article/content as clean Markdown, stripping ads, navigation, and noise (~80% fewer tokens than raw HTML)
Native Markdown fast-path — detects and uses
text/markdownresponses from servers supporting content negotiation (e.g. Cloudflare Markdown for Agents) for instant, no-browser resultsStatic HTML extraction — uses trafilatura to extract content from plain HTTP responses when native Markdown isn't available
Headless browser rendering — falls back to a real Chromium browser (via patchright) for JavaScript-required pages or bot-protected sites, bypassing most Cloudflare challenges
Headed (visible) browser mode — set
headless=falseto use a visible Chromium window to defeat aggressive bot detection (Cloudflare Bot Fight Mode, Turnstile, PerimeterX, DataDome, etc.) that blocks even headless browsersPersistent browser session — the Chromium instance stays alive across calls, so only the first browser-tier fetch pays the cold-start cost (~2–5s)
Content stabilization polling — polls the DOM until extracted content stabilizes across consecutive polls, avoiding premature returns on slow SPAs
Configurable behavior — tune
wait_until,timeout_ms,poll_budget_ms, andpoll_interval_msfor navigation timing and content stabilizationMCP client integration — designed for easy setup with Claude Desktop, LM Studio, Cursor, VS Code, and other MCP clients
Supports Cloudflare's Markdown for Agents protocol, enabling fast markdown retrieval from Cloudflare-hosted sites via content negotiation.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@web-to-markdown-mcp fetch https://example.com/blog/post as markdown"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
web-to-markdown-mcp
An MCP server that fetches a URL and returns the main content as clean Markdown. Uses plain HTTP when possible and real Chromium when needed.
Why
Most MCP web-fetch tools either:
use plain HTTP, which fails on JS-required pages and gets blocked by Cloudflare bot detection on many sites; or
use a real browser but return raw HTML or accessibility-tree snapshots, which are noisy and token-heavy when you just want to read the article.
This server uses a three-tier strategy, using the fastest approach that works for each URL:
Native markdown fast-path. Every request first tries a plain HTTP GET with an
Accept: text/markdownheader. Servers that support content negotiation — such as Cloudflare-hosted sites with Markdown for Agents enabled — respond withContent-Type: text/markdown, and the body is returned immediately with no browser overhead.Plain HTTP + extraction. If the server returns HTML, trafilatura extracts the article body as clean Markdown directly from the static response. Heuristics detect JS shells — pages that return only a "JavaScript required" stub in the static HTML — and fall through to tier 3 for those.
Headless browser. When tiers 1 and 2 yield nothing usable, patchright (a Playwright fork with anti-detection patches) drives real Chromium and trafilatura extracts the rendered content. A single headless browser instance is kept alive across calls — it launches lazily on the first browser-tier fetch and stays running for the session, so subsequent fetches pay only navigation time rather than Chromium startup cost (~2–5 s).
For sites that block even headless patchright (aggressive bot detection), pass headless=False. This uses a visible Chromium window — slower and visually intrusive, but clears most remaining challenges. A headed browser is launched and closed for each fetch that needs it; it is not kept persistent since headed fetches are an infrequent escape hatch rather than the common case.
After browser navigation, the server polls the DOM and runs trafilatura, returning as soon as two consecutive polls produce the same extraction. This means it returns within a few hundred milliseconds for typical pages — rather than waiting for analytics, ads, and other late-loading resources to finish — and gives slow SPAs and bot-challenge clearance time to settle without timing out prematurely.
For a typical article, expect roughly 80% fewer tokens than the raw HTML and roughly 90% fewer than a full accessibility-tree snapshot.
Related MCP server: markfetch
Installation
Requires Python 3.10+ and a one-time Chromium download (~300 MB).
# Run directly with uv (no install step)
uvx web-to-markdown-mcp
# Or install with pip
pip install web-to-markdown-mcp
# One-time browser download
patchright install chromiumMCP client configuration
Claude Desktop
Edit your config file:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.jsonWindows:
%APPDATA%\Claude\claude_desktop_config.json
{
"mcpServers": {
"web-to-markdown": {
"command": "uvx",
"args": ["web-to-markdown-mcp"]
}
}
}Restart Claude Desktop.
LM Studio
Edit ~/.lmstudio/mcp.json (Developer tab → Edit mcp.json) — same JSON block as above. Then enable Allow calling servers from mcp.json in the Developer tab's Server Settings. The server appears in the Integrations tab of any new chat.
Cursor / Windsurf / VS Code
Same JSON block, in each client's MCP config location.
Claude Code
claude mcp add web-to-markdown -- uvx web-to-markdown-mcpUsage
The server exposes a single tool:
fetch_url_as_markdown
Parameter | Type | Default | Description |
| string | required | The URL to fetch |
| string |
| When navigation completes: |
| int |
| Navigation-step timeout in milliseconds |
| bool |
|
|
| int |
| Max time after navigation to wait for content stabilization |
| int |
| How often to re-attempt extraction during polling |
Returns Markdown as a string, or a string beginning with "ERROR:" on expected failures (timeout, no extractable content, navigation error).
Example call (from any MCP client's tool-use UI):
fetch_url_as_markdown(url="https://example.com/long-article")wait_until choice:
"domcontentloaded"(default) — returns when the DOM is built; content-stabilization polling handles the rest"load"— waits for all subresources (images, scripts, stylesheets); rarely needed since polling runs after this"networkidle"— waits for network to quiet; sometimes hangs on pages with persistent background connections"commit"— returns as soon as the response starts; rarely useful
When to bump poll_budget_ms: the 5-second default is fine for typical pages but may return a partial extraction on slow SPAs that render content over many seconds, and may time out before a bot-detection challenge clears in headed mode. For headed-mode fetches of bot-protected sites, 10000-30000 is a reasonable budget.
Limitations
First browser-tier fetch per session. The headless browser launches lazily on the first fetch that needs it (~2–5 s). Subsequent browser-tier fetches in the same session reuse the running instance and pay only navigation time.
Bot detection on hard sites. patchright headless clears default Cloudflare configurations on many sites, but pages running aggressive Bot Fight Mode, Turnstile interactive challenges, or commercial bot-management products (PerimeterX, DataDome, Kasada) — and Cloudflare's own marketing site — still detect headless Chromium. Pass
headless=Falseto use a visible browser window, which clears most of these. The cost is a Chromium window flashing on screen for a couple of seconds per fetch.Headed-mode bot challenges need a generous polling budget. When using
headless=Falseon bot-protected sites, the challenge can take 5-15 seconds to clear. Setpoll_budget_msto 10000-15000 for these cases — the 5000 default may return prematurely while the challenge is still resolving.Headed mode requires a display.
headless=Falsefails on servers without a graphical environment (cloud VMs, containers, CI). Use a virtual display like Xvfb if you need headed mode in those environments.Datacenter IPs. Cloudflare's harder challenges still block requests from datacenter IPs (Oracle Cloud, AWS, etc.) regardless of browser fingerprint or headed/headless mode. Best results come from running this on a residential connection.
Slow SPA rendering. For pages that progressively render content over many seconds, bump
poll_budget_ms. The default returns the most-recent extraction at budget expiry, which on a still-rendering SPA may be partial.Auth-walled content. This server uses a clean browser context with no cookies or stored auth. Logged-in pages won't work.
Comparison with related tools
@playwright/mcp(Microsoft) — general-purpose interactive browser automation: navigate, click, fill forms, run JS, take accessibility snapshots. Use it when you need to interact with a page. Use this server when you need to read a page.mcp-server-fetchand similar HTTP-based servers — also try plain HTTP first, so for static sites the behaviour is similar. The difference is JS-rendered pages and bot-protected sites: those tools don't have a browser fallback, so they fail where this server succeeds.LM Studio Hub plugins like
vadimfedenko/visit-website-reworkedandnpacker/web-tools— same idea inside LM Studio's plugin system. This server runs in any MCP client.
Roadmap
Persistent headless browser instance to amortize cold start
Accept: text/markdownfast path for Cloudflare's Markdown for AgentsOptional
selectorparameter to scope extractionOptional viewport, user-agent, and locale overrides
Optional persistent browser context for sticky cookies / WAF trust
channel="chrome"option to use installed Google Chrome instead of bundled Chromium (further stealth for the hardest sites)
Contributing
Issues and PRs welcome. For substantive changes, please open an issue first to discuss the approach.
License
MIT — see LICENSE.
Available Tools
1 toolfetch_url_as_markdownA
Fetch a URL and return the main content as Markdown.
First tries a plain HTTP request with an Accept: text/markdown header. If the server responds with Content-Type: text/markdown (e.g. Cloudflare Markdown for Agents sites), the body is returned immediately without launching a browser.
Otherwise, uses patchright (a Playwright fork with anti-detection patches) to drive real Chromium, which clears most Cloudflare bot challenges and renders JavaScript-required pages. A single headless Chromium instance is kept alive across calls so subsequent fetches avoid the browser cold-start cost (~2-5s). After navigation, polls the page DOM and runs trafilatura, returning as soon as the extracted Markdown stabilizes across two consecutive polls — typically within a few hundred milliseconds of the DOM being built, regardless of whether trackers, ads, and analytics are still loading in the background.
Inputs: url (required), wait_until, timeout_ms, headless, +2 more. Returns: result
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 1 tool
Only one tool exists, so there is no possibility of confusion between tools.
The single tool is named clearly with a verb_noun pattern (fetch_url_as_markdown), which is descriptive and consistent.
One tool is appropriate for this focused server's purpose; it encapsulates all necessary functionality for fetching a URL and converting it to markdown.
The tool fully covers the server's domain by handling both simple HTTP and JavaScript-heavy pages, returning markdown content without obvious gaps.
Maintenance
Related MCP Connectors
Fetch any URL and get clean Markdown. Web scraping for AI agents.
Fetch a URL and get clean Markdown with metadata. No API key required; rate-limited per IP.
Converts any URL to clean, LLM-ready Markdown using real Chrome browsers
PDFs, JavaScript pages and ordinary HTML as clean Markdown. Follows robots.txt per RFC 9309.
Related MCP Servers
- AlicenseAqualityBmaintenanceGive your AI the ability to read the web. Fetches URLs as clean markdown with 9 fallback strategies.5230 npm9MIT
- AlicenseAqualityCmaintenanceFetches any URL and returns clean markdown, using a real Chrome fingerprint to bypass bot detection. Integrates as an MCP server with tools like fetch_markdown.199 npm5MIT
- FlicenseNot gradedqualityDmaintenanceFetches webpages and returns clean, structured Markdown with metadata (title, author, publish date, description, domain, word count).1-
- AlicenseNot gradedqualityDmaintenanceFetches and renders web pages using a headless Chromium browser, returning clean Markdown or HTML content even for JavaScript-heavy single-page applications.354 npmMIT