🎩 HatFetch

Give any LLM agent the power to read the modern web — JavaScript sites and most bot-protected pages.

Clean Markdown out. Renders JS, rotates residential IPs, escalates to a real stealth browser only when needed.

HatFetch is a Model Context Protocol server that gives Claude, Cursor, and any MCP client the tools scrape, crawl, and screenshot — turning web pages into clean, LLM-ready Markdown.

The difference from a plain fetch: HatFetch escalates automatically. It starts with a fast HTTP fetch; if the page is a JavaScript app that renders client-side, or it gets blocked by bot detection, HatFetch transparently escalates to a real stealth browser (Patchright) running through residential proxies. Your agent stops getting empty shells and Access Denied, and starts getting content.

What it does and doesn't do (honest version)

No tool passes 100% of anti-bot in 2026 — anyone claiming otherwise is selling something. HatFetch gets you the calm-to-medium web reliably and a good share of the hard web, and is honest about the rest.

Quick start

Add HatFetch to your MCP client — no install step, npx handles it:

// Claude Desktop: claude_desktop_config.json  ·  Cursor: ~/.cursor/mcp.json
{
  "mcpServers": {
    "hatfetch": {
      "command": "npx",
      "args": ["-y", "hatfetch"]
    }
  }
}

For Claude Code:

claude mcp add hatfetch -- npx -y hatfetch

That's it. Ask your agent to "read https://example.com and summarize it" and it will use the scrape tool.

Related MCP server: markfetch-mcp

Getting blocked? 🚧

Out of the box HatFetch fetches directly from your machine's IP. That's fine for open sites — but most valuable pages sit behind bot detection that blocks datacenter and server IPs on sight. When that happens, HatFetch tells your agent exactly why, and how to fix it in one step: route through residential IPs.

{
  "mcpServers": {
    "hatfetch": {
      "command": "npx",
      "args": ["-y", "hatfetch"],
      "env": {
        "PROXYHAT_API_KEY": "your-proxyhat-api-key"
      }
    }
  }
}

That's the whole setup. Drop in your API key and HatFetch automatically looks up an active residential sub-user on your account, connects through the gateway, and rotates a fresh IP on every request — retrying automatically when a site pushes back. Get a key (free trial) at proxyhat.com: 50M+ residential & mobile IPs across 148+ countries.

• Explicit gateway credentials — skip the API lookup and use a specific sub-user's proxy login:

  { "env": { "PROXYHAT_USERNAME": "your-proxy-username", "PROXYHAT_PASSWORD": "your-proxy-password" } }

• Pick a specific sub-user while still using the API key: add "PROXYHAT_SUBUSER": "<uuid or name>".

• Bring your own proxy — any HTTP(S) proxy: "PROXY_URL": "http://user:pass@host:port".

Tools

scrape

Fetch a single URL and return its main content as Markdown.

crawl

Breadth-first crawl a site and return every page as Markdown.

screenshot

Render a URL in a real browser (residential + stealth) and return a PNG image.

Browser mode downloads a Chromium build (~150MB) on first use — one-time, automatic. The HTTP path needs no browser.

Why HatFetch

Configuration

All configuration is via environment variables.

HatFetch connects to the ProxyHat HTTP gateway (gate.proxyhat.com:8080) and builds the targeting username for you. Prefer full API-driven provisioning (minting connection URLs via POST /v1/proxy-descriptors)? Set PROXY_URL to the minted URL instead.

Verify your setup

Run the built-in self-test to confirm your install and proxy credentials work end-to-end — it compares your direct IP against the proxied exit IP and checks rotation:

PROXYHAT_API_KEY=your-key npx -y hatfetch --selftest

  ✓ Direct connection works — your IP is 203.0.113.5
  ✓ Proxy resolved — ProxyHat residential (via API key)
  ✓ Traffic is routed through the proxy — exit IP is 183.88.219.209 (not your 203.0.113.5)
  ✓ IP rotation works — second request exited from 73.149.15.4
  ✓ Scrape pipeline works — fetched and parsed example.com

Other commands: hatfetch --version, hatfetch --help.

How it works

HatFetch escalates from cheap to powerful, only paying for what a page needs:

1. HTTP fetch with a realistic User-Agent (direct or through your proxy). Detects blocks — anti-bot status codes (403/429/503) and challenge/block-page signatures from the major vendors (Cloudflare, DataDome, PerimeterX, Incapsula, Akamai) — and retries through a fresh residential IP.

2. Auto-escalate to the browser when the page is an empty JS shell, or was blocked/reset. A real Patchright Chromium (patched to defeat headless fingerprinting) loads the page through the residential proxy, waits for the network to settle, and scrolls to trigger lazy content.

3. Extract the main content with Mozilla Readability and convert to Markdown with Turndown.

4. Be honest — if even the browser lands on a challenge/block page, HatFetch returns an actionable error instead of passing the block page off as content.

Safety: a built-in SSRF guard refuses internal/loopback/metadata addresses, and tool output is size-capped so a huge page can't blow your context window.

Development

npm install
npm run build      # compile to dist/
npm test           # vitest
npm run dev        # run from source (tsx)

Roadmap

• search tool (query → results → scrape)

• Structured extraction (CSS / schema)

• Page actions (click, fill, login flows)

• Optional CAPTCHA-solver hook for the hardest targets

• Python edition

License

MIT © ProxyHat