Skip to main content
Glama
Sign In
Stealth Web Scraper MCP

Stealth Web Scraper MCP

by markkkkkkkkkkkkkkkk
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

Stealth Web Scraper MCP

An MCP server that fetches any web page and returns clean, token-efficient Markdown for AI agents.

  • Tiered fetching, cheapest first: direct HTTP, browser, HTTP via residential proxy, browser via proxy. Each tier runs only when the previous one is blocked.

  • Fast path: curl_cffi with real browser TLS fingerprints (Chrome, Safari, Firefox, Edge), rotated on retry.

  • Browser fallback: when a page is blocked or needs JavaScript, it is rendered in your installed Google Chrome (or Playwright's Chromium).

  • Fewer tokens: pass query to get only the passages that answer a question, ranked with BM25. Long pages are paginated.

  • Safe by default: refuses localhost and private-network URLs, binds to 127.0.0.1, optional API key.

Benchmark

Measured on 30 real sites (docs, reference, news, shops, JavaScript apps and bot-protected sites), with no proxy. Each site was fetched once. Full results are in benchmark/RESULTS.md; re-run with PYTHONPATH=src python benchmark/run.py.

Result

Success rate, no proxy

24/30 (80%)

Tokens vs raw HTML, clean Markdown

97.9% fewer (median 90,222 → 1,901)

Tokens vs raw HTML, with query

99.6% fewer (median 341)

Median time

2.6 s HTTP, 11.4 s browser

The 6 misses: three sites behind strict bot protection that need a residential proxy (Etsy, G2, Glassdoor), and three image-heavy homepages with little text.

Related MCP server: SmartFetch

Install

Claude Code:

claude mcp add stealth-scraper -- uvx stealth-scraper-mcp

Claude Desktop, Cursor, VS Code and other clients (mcpServers config):

{
  "mcpServers": {
    "stealth-scraper": {
      "command": "uvx",
      "args": ["stealth-scraper-mcp"]
    }
  }
}

The browser fallback uses Google Chrome if it is installed. Otherwise download Chromium once:

uvx stealth-scraper-mcp --install-browser

Tool: scrape_url_stealth

Parameter

Default

Purpose

url

required

Page to fetch

query

none

Return only the passages relevant to this question

top_k

5

Passages to return with query

output_format

markdown

markdown, text, html or links

response_format

concise

detailed adds engine, metadata and cache diagnostics

max_chars / offset

15000 / 0

Pagination; pass next_offset back as offset

max_age_seconds

1 hour

Cache freshness; 0 forces a fresh fetch, -1 reads only the cache

render_js

auto

always or never force or skip the browser

proxy

auto

auto uses the residential proxy only after a block; never or always

css_selector

none

Only return matching elements

wait_for_selector, scroll

none / false

Browser options for dynamic and lazy-loaded pages

Errors come back as [code] message Hint: ..., e.g. blocked, timeout, url_not_allowed, selector_not_found.

Running over HTTP

uvx stealth-scraper-mcp --transport http --port 8080

The endpoint is http://127.0.0.1:8080/mcp, with /health for checks. To expose it on a network, set an API key first:

SCRAPER_API_KEY=change-me SCRAPER_HOST=0.0.0.0 uvx stealth-scraper-mcp --transport http

Clients then send Authorization: Bearer change-me.

Configuration

Variable

Default

Purpose

SCRAPER_PROXY_URL

none

Residential proxy, used only after a direct fetch is blocked

SCRAPER_API_KEY

none

Require a bearer token on HTTP

SCRAPER_HOST / SCRAPER_PORT

127.0.0.1 / 8080

HTTP bind address

SCRAPER_CACHE_TTL

3600

Default cache freshness in seconds

SCRAPER_CACHE_MAX_ENTRIES

256

Cache size

SCRAPER_BROWSER_CHANNEL

chrome

Browser to launch; empty for bundled Chromium

SCRAPER_BROWSER_HEADLESS

true

false runs a visible, harder-to-detect browser

SCRAPER_BROWSER_LOCALE / SCRAPER_BROWSER_TIMEZONE

en-US / system

Keep these consistent with your proxy's location

SCRAPER_ALLOW_PRIVATE_NETWORKS

false

Allow localhost and private IPs

SCRAPER_FETCH_TIMEOUT, SCRAPER_MAX_RETRIES

20, 2

Request timeout and retries

Plugins

Packages can extend the server through the stealth_scraper_mcp.plugins entry point group. They can add usage metering, extra HTTP routes and CLI commands; see src/stealth_scraper_mcp/plugins.py. Without plugins the server is free and unmetered.

Responsible use

Respect each site's terms of service and robots.txt, and avoid heavy request rates.

License

AGPL-3.0-or-later. You can use, modify and self-host this freely. If you offer a modified version to others as a network service, you must publish your changes under the same license.

Available Tools

1 tool
scrape_url_stealthScrape URLA
Read-onlyIdempotent

Fetch a web page and return its main content as clean Markdown.

Pass query to get only the passages that answer a question. Handles JavaScript-rendered and bot-protected pages by falling back to a real Chrome browser. Results are cached; long pages are paginated with offset/next_offset.

Inputs: url (required), proxy, query, top_k, +11 more. Returns: hint, cache, title, engine, +15 more

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Deploy Server
    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    TDQS

    A4.1/5.0

    Scored across 1 tool

    Disambiguation5/5

    With a single tool there is no possibility of selection confusion; an agent cannot mistake scrape_url_stealth for anything else. The description clearly scopes the tool's behavior (query filtering, JS fallback, caching, pagination).

    Naming Consistency5/5

    The lone name scrape_url_stealth follows a clear verb_noun_modifier snake_case pattern. No competing conventions exist to create inconsistency.

    Tool Count3/5

    One tool is thin for a server branded as a scraper platform, even though that tool is genuinely capable. It lands in the borderline zone where a single entry point handles everything but leaves no room for distinct capabilities.

    Completeness3/5

    Core single-page fetch, JS rendering, caching, and pagination are covered, but common scraping needs are absent: batch/multi-URL fetch, crawling, sitemap/robots handling, and structured or raw HTML output modes. Agents can work around these, but the surface is not fully complete for the stated domain.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Resources

    Related MCP Connectors

    Related MCP Servers

    • SmartFetch

      Friezaaaa
      F
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to reliably fetch public web URLs and extract clean, agent-ready text, Markdown, links, and metadata, with automatic browser fallback and optional x402 payment support.
      -
    • JsonLink MCP Server

      JsonLink MCP Serverofficial

      mauriblint
      A
      license
      A
      quality
      C
      maintenance
      Enables AI agents to read any web page as clean Markdown, pull title/description/preview metadata, capture desktop, mobile, or full-page screenshots, and list every link or image on a URL. JavaScript-heavy pages are rendered in a real browser and blocked requests retried through a proxy, so agents can reach content they'd otherwise miss.
      5
      50 npm
      MIT

    • read-website-fast

      cCastillo2488
      F
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents to quickly fetch webpages, strip noise, and convert content into clean, link-preserving Markdown with caching and optional multi-page crawling for token-efficient web scraping.
      -