A
licenseA
qualityD
maintenanceEnables AI agents to fetch any web page as clean markdown or screenshot it, turning URLs into LLM-ready context.
2
5 npm
MIT
Stealth Web Scraper MCP
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Stealth Web Scraper MCPscrape stripe.com pricing and tell me what the Pro plan costs"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Stealth Web Scraper MCP
An MCP server that fetches any web page and returns clean, token-efficient Markdown for AI agents.
Tiered fetching, cheapest first: direct HTTP, browser, HTTP via residential proxy, browser via proxy. Each tier runs only when the previous one is blocked.
Fast path:
curl_cffiwith real browser TLS fingerprints (Chrome, Safari, Firefox, Edge), rotated on retry.Browser fallback: when a page is blocked or needs JavaScript, it is rendered in your installed Google Chrome (or Playwright's Chromium).
Fewer tokens: pass
queryto get only the passages that answer a question, ranked with BM25. Long pages are paginated.Safe by default: refuses localhost and private-network URLs, binds to
127.0.0.1, optional API key.
Benchmark
Measured on 30 real sites (docs, reference, news, shops, JavaScript apps and bot-protected sites), with no proxy. Each site was fetched once. Full results are in benchmark/RESULTS.md; re-run with PYTHONPATH=src python benchmark/run.py.
Result | |
Success rate, no proxy | 24/30 (80%) |
Tokens vs raw HTML, clean Markdown | 97.9% fewer (median 90,222 → 1,901) |
Tokens vs raw HTML, with | 99.6% fewer (median 341) |
Median time | 2.6 s HTTP, 11.4 s browser |
The 6 misses: three sites behind strict bot protection that need a residential proxy (Etsy, G2, Glassdoor), and three image-heavy homepages with little text.
Related MCP server: SmartFetch
Install
Claude Code:
claude mcp add stealth-scraper -- uvx stealth-scraper-mcpClaude Desktop, Cursor, VS Code and other clients (mcpServers config):
{
"mcpServers": {
"stealth-scraper": {
"command": "uvx",
"args": ["stealth-scraper-mcp"]
}
}
}The browser fallback uses Google Chrome if it is installed. Otherwise download Chromium once:
uvx stealth-scraper-mcp --install-browserTool: scrape_url_stealth
Parameter | Default | Purpose |
| required | Page to fetch |
| none | Return only the passages relevant to this question |
| 5 | Passages to return with |
|
|
|
|
|
|
| 15000 / 0 | Pagination; pass |
| 1 hour | Cache freshness; |
|
|
|
|
|
|
| none | Only return matching elements |
| none / false | Browser options for dynamic and lazy-loaded pages |
Errors come back as [code] message Hint: ..., e.g. blocked, timeout, url_not_allowed, selector_not_found.
Running over HTTP
uvx stealth-scraper-mcp --transport http --port 8080The endpoint is http://127.0.0.1:8080/mcp, with /health for checks. To expose it on a network, set an API key first:
SCRAPER_API_KEY=change-me SCRAPER_HOST=0.0.0.0 uvx stealth-scraper-mcp --transport httpClients then send Authorization: Bearer change-me.
Configuration
Variable | Default | Purpose |
| none | Residential proxy, used only after a direct fetch is blocked |
| none | Require a bearer token on HTTP |
|
| HTTP bind address |
|
| Default cache freshness in seconds |
|
| Cache size |
|
| Browser to launch; empty for bundled Chromium |
|
|
|
|
| Keep these consistent with your proxy's location |
|
| Allow localhost and private IPs |
|
| Request timeout and retries |
Plugins
Packages can extend the server through the stealth_scraper_mcp.plugins entry point group. They can add usage metering, extra HTTP routes and CLI commands; see src/stealth_scraper_mcp/plugins.py. Without plugins the server is free and unmetered.
Responsible use
Respect each site's terms of service and robots.txt, and avoid heavy request rates.
License
AGPL-3.0-or-later. You can use, modify and self-host this freely. If you offer a modified version to others as a network service, you must publish your changes under the same license.
Available Tools
1 toolscrape_url_stealthScrape URLARead-onlyIdempotent
Fetch a web page and return its main content as clean Markdown.
Pass query to get only the passages that answer a question. Handles JavaScript-rendered and bot-protected pages by falling back to a real Chrome browser. Results are cached; long pages are paginated with offset/next_offset.
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Absolute http(s) URL to fetch | |
| proxy | No | auto (default) retries through the residential proxy only when blocked, since proxy bandwidth is paid. never avoids it; always skips the direct attempt | auto |
| query | No | Question or keywords; returns only the most relevant passages instead of the whole page. Strongly recommended for long pages. | |
| top_k | No | With query: number of passages to return | |
| offset | No | Character offset to start from; use next_offset from a previous call | |
| scroll | No | Browser only: scroll down to trigger lazy-loaded content | |
| max_chars | No | Maximum characters of content to return (about 4 chars per token) | |
| render_js | No | auto (default) uses a headless browser only when the fast fetch is blocked or nearly empty | auto |
| css_selector | No | Only return elements matching this CSS selector, e.g. 'article' or '#pricing table' | |
| include_links | No | Keep hyperlinks in markdown output; false saves tokens | |
| output_format | No | markdown (default): main content; text: plain text; html: cleaned HTML; links: every link on the page | markdown |
| max_age_seconds | No | Accept a cached copy up to this old. 0 always fetches fresh; -1 uses only the cache. Default: server setting (1 hour). | |
| response_format | No | concise (default) omits fetch diagnostics and page metadata; detailed includes them | concise |
| wait_for_selector | No | Browser only: wait until this CSS selector appears before reading the page | |
| impersonate_browser | No | TLS fingerprint to present first; others are tried on retry | chrome |
Output Schema
ParametersJSON Schema
| Name | Required | Description |
|---|---|---|
| hint | No | How to narrow or continue the request, when relevant |
| cache | Yes | Whether the page came from the local cache |
| title | No | Page title, when detected |
| engine | No | Fingerprint or browser that served the page |
| content | Yes | The requested window of extracted content |
| cost_usd | No | Amount charged for this call; failures are free |
| metadata | No | Description, author, site name, date and language |
| final_url | Yes | URL after redirects |
| escalation | No | Which cheaper tiers were tried first and why they failed |
| est_tokens | Yes | Approximate tokens in content (chars / 4) |
| next_offset | No | Pass as offset to read more; null when nothing remains |
| proxy_bytes | No | Residential proxy bandwidth used by this fetch |
| status_code | Yes | HTTP status code of the final page |
| total_chars | Yes | Length of the full (or query-filtered) content |
| content_hash | No | SHA-256 of the full extracted content |
| fetch_method | No | Fast HTTP client or headless browser |
| passages_matched | No | Number of passages kept by query |
| cache_age_seconds | No | Age of the fetch this result was built from |
| credits_remaining_usd | No | Balance left on this API key |
TDQS
A4.2/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, openWorld, so the bar is lower. The description still adds genuine behavioral context beyond them: automatic fallback to a real Chrome browser for JS/bot-protected pages, caching of results, and pagination semantics. It omits cost/rate implications of proxy usage, which the schema covers instead.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, each front-loaded with a distinct fact (core behavior, query mode, JS fallback, caching/pagination). No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 15-parameter tool with a full schema and an output schema, the description covers the essential mechanics an agent needs: what it returns, how to narrow results, JS fallback, and pagination. Remaining details (proxy cost, output formats, cache age) are handled by the schema descriptions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all 15 parameters are already well documented, setting the baseline at 3. The description highlights query and offset/next_offset meaningfully, but largely restates what the schema already explains.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Fetch a web page and return its main content as clean Markdown'), plus the key transformation (main content extraction to Markdown). No siblings exist to differentiate from, and the scope is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear conditional guidance: pass query to get answer passages, use offset/next_offset for long pages, and notes caching. It does not explicitly say when not to use it, but with no sibling tools that exclusion is largely moot.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
1 tool update
v0.1.0- First observed
scrape_url_stealth
TDQS
A4.1/5.0
Scored across 1 tool
Disambiguation5/5
With a single tool there is no possibility of selection confusion; an agent cannot mistake scrape_url_stealth for anything else. The description clearly scopes the tool's behavior (query filtering, JS fallback, caching, pagination).
Naming Consistency5/5
The lone name scrape_url_stealth follows a clear verb_noun_modifier snake_case pattern. No competing conventions exist to create inconsistency.
Tool Count3/5
One tool is thin for a server branded as a scraper platform, even though that tool is genuinely capable. It lands in the borderline zone where a single entry point handles everything but leaves no room for distinct capabilities.
Completeness3/5
Core single-page fetch, JS rendering, caching, and pagination are covered, but common scraping needs are absent: batch/multi-URL fetch, crawling, sitemap/robots handling, and structured or raw HTML output modes. Agents can work around these, but the surface is not fully complete for the stated domain.
Maintenance
ActivityMaintained
ResponsivenessNo issues
Related MCP Connectors
Read any web page as clean Markdown for AI agents: fetch, search, metadata, links. SSRF-safe.
Reliable web access for AI agents: smart HTTP, rotating proxies, and full-browser rendering.
Fetch pages as markdown, search web and news, extract structured data. For AI agents.
Web search and any page as LLM-ready Markdown for AI agents. Free keyless tier, no signup.
131
Related MCP Servers
- FlicenseNot gradedqualityBmaintenanceEnables AI agents to reliably fetch public web URLs and extract clean, agent-ready text, Markdown, links, and metadata, with automatic browser fallback and optional x402 payment support.-
- AlicenseAqualityBmaintenanceEnables reading web pages and PDFs as clean Markdown, extracting main content and letting agents query for relevant passages, outline sections, search in-page, and list links while respecting robots.txt and blocking private addresses.41MIT
JsonLink MCP Serverofficial
AlicenseAqualityCmaintenanceEnables AI agents to read any web page as clean Markdown, pull title/description/preview metadata, capture desktop, mobile, or full-page screenshots, and list every link or image on a URL. JavaScript-heavy pages are rendered in a real browser and blocked requests retried through a proxy, so agents can reach content they'd otherwise miss.5MIT