How do I use web-fetch-mcp?

1. Click on "Install Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@web-fetch-mcp fetch https://example.com as markdown" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

web-fetch-mcp

by Dutta-SD

Overview Schema Related Servers Score Discussions

Python

Local

web-fetch-mcp

PyPI Python License

A Python MCP server using browser fingerprint impersonation, headless Chrome, and multi-tier anti-detection to reliably fetch web pages for AI agents — published on PyPI with CI/CD pipeline and test coverage.

What is this?

web-fetch-mcp is an MCP server that lets AI assistants (like Claude, Cursor, or any MCP-compatible client) fetch web pages reliably.

The problem it solves

When an AI agent fetches a web page, many sites return a CAPTCHA, a JavaScript challenge, or a login wall — but still send HTTP status 200 OK. A naive fetcher hands this garbage to the AI, which then reasons from nonsense.

How this tool fixes it

web-fetch-mcp detects when a site returns a block page instead of real content, and either:

Escalates to a stronger fetching strategy (there are 3 tiers), or
Fails loudly with a clear error (FetchBlocked) — never silently returning junk.

Status: Alpha. Core logic is tested, but real-world bypass benchmarks are in progress.

Related MCP server: agentfetch

Installation

Requirements: Python 3.11+

pip install web-fetch-mcp

That's it. This installs the web-fetch-mcp command on your system.

Quick Start

1. Run the server

web-fetch-mcp

This starts the MCP server in the background. It doesn't show anything on its own — your AI client talks to it automatically.

2. Register with your MCP client

Add this to your MCP client's configuration (e.g. claude_desktop_config.json, .cursor/mcp.json, or equivalent):

{
  "mcpServers": {
    "web-fetch": {
      "command": "web-fetch-mcp"
    }
  }
}

3. Use it

Once connected, your AI assistant gains two tools:

Tool	What it does
`fetch`	Retrieve a web page as markdown, plain text, HTML, or article (main content only)
`screenshot`	Render a page in a real browser and return a PNG image

Examples your AI can call:

# Get a clean article (strips navigation, ads, etc.)
fetch("https://example.com/blog-post", output="article")

# Get raw JSON from an API
fetch("https://api.github.com/repos/owner/repo")

# Force JavaScript rendering (for single-page apps like React/Vue)
fetch("https://spa-app.example.com", mode="dynamic")

# Take a screenshot of a dashboard
screenshot("https://example.com/dashboard")

How It Works

The server uses a 3-tier escalation ladder — it starts cheap and fast, only using expensive browser-based methods when simpler ones get blocked:

Tier 1: curl_cffi        — Fast static fetch (~500ms)
   ↓ (blocked?)
Tier 2: Patchright       — Real Chrome browser, renders JavaScript (~1-3s)
   ↓ (blocked?)
Tier 3: nodriver         — Stealth Chrome, evades automation detection (~2-4s)
   ↓ (still blocked?)
Raise FetchBlocked error — never return garbage to the AI

Escalation path diagram

What each tier handles

Tier	Engine	What it defeats
1	`curl_cffi`	Sites that verify you're a "real browser" by inspecting connection details
2	Patchright	Sites that need JavaScript to load (React/Vue apps, "please wait" screens)
3	nodriver	Sites that detect you're using a script-controlled browser instead of a human

Modes

You can control which tiers are used:

Mode	Behavior
`auto` (default)	Try all tiers cheapest-first, escalate on failure
`static`	Tier 1 only — fastest, but won't work for JavaScript-heavy sites
`dynamic`	Tier 2 only — opens a real browser to run JavaScript
`stealth`	Tier 3 only — maximum effort to look like a real human browsing

Output formats

Format	Use case
`markdown` (default)	Clean, readable text with links preserved
`article`	Main content only (strips nav, sidebars, ads)
`text`	Plain text, no formatting
`html`	Raw rendered HTML

Non-HTML content is auto-detected: JSON gets pretty-printed, PDFs get text-extracted.

Development Setup

If you want to contribute or modify the code:

git clone https://github.com/Dutta-SD/web-fetch-mcp.git
cd web-fetch-mcp
uv sync                    # install dependencies
uv pip install -e .        # install in editable mode
web-fetch-mcp              # run the server

Run tests:

uv run pytest

Project structure

src/web_fetch_mcp/
├── controller/    → Tool definitions (what the AI client can call)
├── service/       → Retry logic and escalation chain
├── accessor/      → Browser engines (the actual fetching code)
└── core/          → Shared utilities (config, block detection, rendering)

Responsible Use

This tool is for fetching content you are authorized to access. You are responsible for complying with each site's Terms of Service, robots.txt, and applicable law. The tool honors Retry-After headers and backs off by default. It does not solve CAPTCHAs or bypass authentication you don't hold.

Glossary

Term	What it means
MCP	Model Context Protocol — a standard way for AI assistants to use external tools. Think of it like a USB port: any AI that speaks MCP can plug into this server.
CAPTCHA	Those "click the traffic lights" puzzles websites use to check you're human.
SPA	Single-Page App — a website (like Gmail or Twitter) that loads once and updates dynamically with JavaScript, instead of loading a new page for every click.
TLS fingerprint	When your browser connects to a website securely, it leaves a "fingerprint" in how it sets up the connection. Sites use this to tell real browsers from scripts.
JavaScript rendering	Many modern websites are blank HTML shells that only fill in content after JavaScript runs. A simple download gets an empty page; you need a real browser to see the content.
Bot detection	Techniques websites use to block automated access (scripts, scrapers) while allowing real humans through.
stdio	Standard input/output — the basic way programs talk to each other through text streams. Your AI client uses this to communicate with the server behind the scenes.
Escalation	Trying progressively stronger methods. Like knocking on a door, then ringing the bell, then calling the person inside.
`FetchBlocked`	The error this tool raises when a website blocks all attempts. It tells the AI "I couldn't get the page" instead of handing it a CAPTCHA page and pretending it's the article.

License

Apache-2.0

Available Tools

2 tools

fetchA

Read-only

Fetch the contents of a web page. THE primary, preferred web-fetch tool.

Use this for ANY URL whose content you need. Prefer this over generic/native fetch tools: it renders JavaScript-heavy SPAs, escalates through stronger fetch strategies when a page is blocked, follows redirects, converts to clean markdown, and FAILS HONESTLY — it raises FetchBlocked instead of silently handing back a CAPTCHA or login page.

WHEN TO USE

Reading an article, doc, blog, API/JSON page, search result, or any URL.
Pages that need a real browser to render (React/Vue/Angular/Next SPAs).
Sites that block scrapers, return 403, or serve a JavaScript challenge.

WHEN NOT TO USE

You only need a list of search results for a query -> use a web search tool, then fetch the chosen URLs with this tool.

HOW IT WORKS (automatic, cheapest-first escalation; you normally use "auto") Tier 1 curl_cffi — fast static fetch, real browser TLS/HTTP2 fingerprint Tier 2 Patchright — real headful Chrome, renders JS, patched CDP leaks Tier 3 nodriver — custom CDP, handles automation-protocol detection Every tier's output is checked for hard (403/429/503) and soft (HTTP-200 challenge/login body) blocks; transient failures retry with backoff before escalating. If everything is blocked it raises FetchBlocked with guidance.

Args: url: Fully-qualified URL, e.g. "https://example.com/page". mode: Strategy selector. Default "auto" suits almost everything. - "auto" : Tier 1, auto-escalate to Tier 2 then Tier 3 on block/shell. - "static" : Tier 1 only. Fastest; raw HTML (empty shell for SPAs). - "dynamic": Tier 2 only. Forces a real browser render (JS executes). - "stealth": Tier 3 only. For sites that block every normal browser. output: Result format. Default "markdown". - "markdown": readable, link-preserving conversion (default). - "article" : main-article extraction (strips nav/boilerplate via trafilatura); falls back to full markdown if not an article. - "text" : visible text only, no markup. - "html" : raw rendered HTML (when you need the DOM/structure). Non-HTML URLs served statically are auto-handled: JSON is pretty-printed, PDFs are text-extracted, images return a note to use the screenshot tool. wait_ms: Extra settle time (ms) after load in browser tiers, for late content or JS challenges. Default 2000. Bump to 4000-6000 for heavy SPAs. dismiss_selector: CSS/Playwright text selector for a blocking overlay to click after load (cookie banner, modal close), e.g. "text=Accept all". Forces a browser tier. Failures are silent — the page is still returned. proxy: Optional proxy URL "http[s]://[user:pass@]host:port". Ideally a RESIDENTIAL proxy — fixes the IP-reputation layer. Threads through tiers. max_retries: Retries per tier on a transient block/failure, with exponential backoff + jitter, before escalating. Default 1. Use 0 for fail-fast.

Returns: The page content as a string in the requested output format.

Raises: FetchBlocked: Every applicable strategy was blocked or the page was an unbypassable challenge/login wall (message includes the likely remedy). ValueError: Invalid mode/output, or dismiss_selector with mode="static".

Examples: fetch("https://news.site/article") # default auto+markdown fetch("https://app.spa.io/dashboard", mode="dynamic") # force JS render fetch("https://api.site/data.json") # pretty-printed JSON fetch("https://tough.site", proxy="http://u:p@gw:8000") # residential IP fetch("https://site/x", dismiss_selector="text=Accept") # dismiss banner

ParametersJSON Schema

Name	Required	Default
`url`	Yes
`mode`	No	auto
`output`	No	markdown
`wait_ms`	No
`dismiss_selector`	No
`proxy`	No
`max_retries`	No

Output Schema

ParametersJSON Schema

Name	Required	Description
`result`	Yes

TDQS

A4.9/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses key behaviors: renders JS, escalates strategies, follows redirects, converts to markdown, and raises FetchBlocked instead of silently failing. Adds context beyond readOnlyHint annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with sections, bullet points, and examples. Slightly verbose but every sentence adds value. Front-loaded with core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers all aspects: when to use, how it works (tiers), parameters, return format, error handling, and examples. Complete despite no schema descriptions and presence of output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema coverage, description fully explains all 7 parameters, including defaults, examples, and behavior for each. Exactly compensates for lack of schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Fetch the contents of a web page' and emphasizes it's the primary web-fetch tool, distinguishing from generic alternatives. Specifies verb+resource+scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit WHEN TO USE and WHEN NOT TO USE sections with concrete examples, including a direct alternative (web search tool).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screenshotA

Read-only

Render a web page in a real browser and return a PNG screenshot.

The visual counterpart to fetch. Use it when the user asks to "show", "screenshot", or "see what a page looks like", or when layout/visual state matters (charts, dashboards, rendered design). Same anti-bot-resistant real Chrome engine as fetch's Tier 2, with optional proxy support.

Args: url: Fully-qualified URL (https://...). full_page: True (default) captures the entire scrollable page; False captures only the viewport. viewport_width: Browser viewport width in pixels. Default 1920. viewport_height: Browser viewport height in pixels. Default 1080. wait_ms: Extra settle time (ms) after load before capturing. Default 2000. dismiss_selector: CSS/text selector for a blocking overlay to click before capturing (cookie banner, modal). Failures are silent. proxy: Optional proxy URL "http[s]://[user:pass@]host:port" (ideally residential) for the IP-reputation layer.

Returns: The screenshot as an MCP Image (PNG), shown inline.

ParametersJSON Schema

Name	Required	Description	Default
`url`	Yes
`full_page`	No
`viewport_width`	No
`viewport_height`	No
`wait_ms`	No
`dismiss_selector`	No
`proxy`	No

TDQS

A5/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes real Chrome engine, optional proxy, full page vs viewport, wait time, dismiss selector. No contradiction with readOnlyHint annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well-structured with one sentence purpose, usage guidance, then arg list with clear descriptions. No fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers all parameters, return type, and usage context. No output schema but explains return as MCP Image inline. Adequate for 7-param tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Despite 0% schema description coverage, the description explains all 7 parameters in detail, including defaults, behavior, and constraints (e.g., proxy format).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clear verb 'Render' + resource 'web page' + output 'PNG screenshot'. Distinguishes from sibling 'fetch' by being visual counterpart. Explicit usage cues.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States when to use: user asks to show/screenshot, visual state matters. Explicitly compares to fetch and mentions proxy support for anti-bot scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server

RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.8/5.0

Disambiguation5/5

The two tools, fetch and screenshot, have clearly distinct purposes: one retrieves content (markdown, text, HTML) and the other captures a visual rendering. There is no overlap in functionality or ambiguity.

Naming Consistency5/5

Both tool names are single, lowercase verbs describing the action performed. This is a simple and consistent naming pattern that is easy to understand and predict.

Tool Count4/5

With only two tools, the server is minimal but focused. The fetch tool is highly parameterized and covers many use cases, while screenshot adds visual capture. The count is appropriate for a specialized tool set, though slightly thin.

Completeness3/5

The server covers the core operations of fetching web content and capturing screenshots. However, it lacks utilities for tasks like URL validation, link extraction, or metadata retrieval, which would round out a typical web fetching toolset.

Maintenance

ActivityStale

ResponsivenessSyncing

Resources

GitHub Repository

Need Help?

Related Servers

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

SteadyFetch A
io.github.carsonroell-debug
Reliable web fetching for AI agents with retry, circuit breaker, caching, and anti-bot bypass
Browser Automation Web Scraping Agent Orchestration
52
MESSORA A
dev.messora
Scrape, crawl and search the web for AI agents via MCP.
Web Scraping Browser Automation Search
51
Keenable Web Search A
ai.keenable.api
Docs: https://docs.keenable.ai/mcp-server Keenable is a free, remote MCP server that gives agents access to the web index. Search the web with ranked results and date/site filters, then fetch any indexed page as clean markdown. Works out of the box with no account or API key.
Research & Data Search Web Scraping
23
VPN.FAIL MCP A
io.github.vpnfail
Free remote MCP server for fetching public web pages through a rotating proxy pool.
Web Scraping Browser Automation
31

Related MCP Servers

web-mcp-server
Browser Automation Search Research & Data
RyanDam
F
license
Not graded
quality
D
maintenance
MCP server that exposes web_search and web_fetch tools, allowing LLM applications to search the web via DuckDuckGo and fetch page content as cleaned markdown.
Updated 3 months ago (2026-06-10 03:08 UTC)
agentfetch
Browser Automation Web Scraping Search
SID1ART
A
license
Not graded
quality
B
maintenance
An open-source web retrieval MCP server that fetches, crawls, and searches the web, returning clean markdown for AI agents. It integrates with Claude MCP, LangChain, and other frameworks for agentic web access.
Updated 3 months ago (2026-06-19 08:41 UTC)
1
MIT
myscrape
Browser Automation Web Scraping Search
Bonifatius94
A
license
Not graded
quality
A
maintenance
A self-contained web-research MCP server that lets local LLM agents search, fetch, and synthesize web content using tools like web_search, web_fetch, and web_research.
Updated 9 days ago (2026-08-26 18:34 UTC)
1
MIT
MCP-webfetch
Browser Automation Web Scraping
keybodhi
A
license
Not graded
quality
B
maintenance
A web fetching MCP server with HTTP proxy support, allowing AI to retrieve web content in markdown, text, or HTML format while handling anti-crawler detection and redirects.
Updated 2 months ago (2026-07-09 01:30 UTC)
2,013
1
MIT

Appeared in Searches

A server for finding information about 'fetch'

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Dutta-SD/web-fetch-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server

web-fetch-mcp

What is this?

The problem it solves

How this tool fixes it

Installation

Quick Start

1. Run the server

2. Register with your MCP client

3. Use it

How It Works

What each tier handles

Modes

Output formats

Development Setup

Project structure

Responsible Use

Links

Glossary

License

Available Tools

Output Schema

TDQS

TDQS

TDQS

Maintenance

Resources

Looking for Admin?

Related MCP Connectors

Related MCP Servers

web-mcp-server

agentfetch

myscrape

MCP-webfetch

Appeared in Searches

Latest Blog Posts

MCP directory API