superFetch MCP Server

One-Click Install

A Model Context Protocol (MCP) server that fetches web pages, extracts readable content with Mozilla Readability, and returns AI-friendly Markdown.

Built for AI workflows that need clean text, stable metadata, and safe-by-default fetching.

Great for: LLM summarization, context retrieval, knowledge base ingestion, and AI agents.

| Quick Start | Tool | Resources | Configuration | Security | Development |

Features

• Cleaner outputs for LLMs: Readability extraction with quality gates (content ratio + heading retention ≥ 70%)

• Markdown that’s easy to consume: metadata footer for HTML + configurable source injection for raw Markdown (markdown or frontmatter)

• Handles “raw content” sources: preserves markdown/text; rewrites GitHub/GitLab/Bitbucket/Gist URLs to raw

• Works for both local and hosted setups:

  • Stdio mode: best for MCP clients (VS Code / Claude Desktop / Cursor)

  • HTTP mode: best for self-hosting (auth, sessions, rate limiting, Host/Origin validation)

• Fast and resilient: redirect validation, timeouts, and response size limits

• Security-first defaults: URL validation + SSRF/DNS/IP blocklists (blocks private ranges and cloud metadata endpoints)

You get, in one tool call:

• Clean, readable Markdown from any public URL (docs, articles, blogs, wikis)

If you’re comparing “just call fetch()” vs superFetch: superFetch focuses on extracting the main content in a readble format for LLMs and even humans, when requested url is fetched it returns clean structured markdown that can also be saved as a resource for later use.

What it is (and isn’t)

• It is a content extraction tool: focuses on extracting readable content, not screenshots or full-page data.

• It is an MCP server: integrates with any MCP-compatible client (Claude Desktop, VS Code, Cursor, Cline, Windsurf, Codex, etc).

• It isn’t a general web scraper: it extracts main content, not all page elements.

• It isn’t a browser: it doesn’t execute JavaScript or render pages.

• It’s opinionated on safety: blocks private/internal URLs and cloud metadata endpoints by default.

Quick Start

Recommended: use stdio mode with your MCP client (no HTTP server).

Try it in 60 seconds

1. Add the MCP server config (below)

2. Restart your MCP client

3. Call the fetch-url tool with any public URL

What the tool returns

You’ll get structuredContent with url, resolvedUrl, optional title, and markdown (plus a superfetch://cache/... resource link when cache is enabled and content is large).

Claude Desktop

Add to your claude_desktop_config.json:

VS Code

Add to .vscode/mcp.json in your workspace:

With Custom Configuration

Add environment variables in your MCP client config under env. See Configuration or CONFIGURATION.md for all available options and presets.

Example output (trimmed)

Tip (Windows): If you encounter issues, try: cmd /c "npx -y @j0hanz/superfetch@latest --stdio"

Cursor

1. Open Cursor Settings

2. Go to Features > MCP Servers

3. Click "+ Add new global MCP server"

4. Add this configuration:

Add to your ~/.codex/config.toml file:

Basic Configuration:

With Environment Variables: See CONFIGURATION.md for examples.

Access config file: Click the gear icon -> "Codex Settings > Open config.toml"

Documentation: Codex MCP Guide

Open the Cline MCP settings file:

macOS:

Windows:

Add the configuration:

Add to ./codeium/windsurf/model_config.json:

macOS:

Windows:

Use cases

1) Turn a docs page into “LLM-ready” Markdown

• Call fetch-url with the docs URL

• Feed the returned markdown into your summarizer / chunker

• Use the metadata footer fields (especially Original Source) for citations

2) Fetch a GitHub/GitLab/Bitbucket file as raw markdown

• Pass the normal “web UI” URL to fetch-url

• superFetch will rewrite it to the raw content URL when possible

• This avoids navigation UI and reduces boilerplate

3) Large pages: keep responses stable with cache resources

• When content is large, the tool can include a superfetch://cache/... resource link

• In MCP clients that support resources, you can read the full content via the resource URI

• In HTTP mode, you can also download cached content via /mcp/downloads/:namespace/:hash when cache is enabled

4) Safe-by-default web access for agents

• superFetch blocks private IP ranges and common cloud metadata endpoints

• If your agent needs internal access, this is intentionally not supported by default (see Security)

Installation (Alternative)

Global Installation

From Source

Running the Server

HTTP mode requires authentication. By default it binds to 127.0.0.1. Non-loopback HOST values require ALLOW_REMOTE=true. To listen on all interfaces, set HOST=0.0.0.0 or HOST=::, set ALLOW_REMOTE=true, and configure OAuth (remote bindings require OAuth).

Windows (PowerShell):

For multiple static tokens, set ACCESS_TOKENS (comma/space separated).

Auth is required for /mcp and /mcp/downloads via Authorization: Bearer <token> (static mode also accepts X-API-Key).

Endpoints:

• GET /health (no auth; returns status, name, version, uptime)

• POST /mcp (auth required)

• GET /mcp (auth required; SSE stream; requires Accept: text/event-stream)

• DELETE /mcp (auth required)

• GET /mcp/downloads/:namespace/:hash (auth required)

Sessions are managed via the mcp-session-id header (see HTTP Mode Details).

Available Tools

Tool Response Notes

The tool returns structuredContent with url, inputUrl, resolvedUrl, optional title, and markdown when inline content is available. resolvedUrl may differ from inputUrl when the URL is rewritten to raw content (GitHub/GitLab/Bitbucket/Gist). On errors, error is included instead of content.

The response includes:

• a text block containing JSON of structuredContent

• a resource block embedding markdown when inline content is available (stdio always embeds full markdown; HTTP embeds inline markdown when it fits or when truncated)

• when content exceeds the inline limit and cache is enabled, a resource_link block pointing to superfetch://cache/... (stdio mode still embeds full markdown; HTTP mode omits embedded markdown)

• error responses set isError: true and return structuredContent with error and url

fetch-url

Fetches a webpage and converts it to clean Markdown format with a metadata footer for HTML (raw markdown is preserved with source injection).

Example

Error response:

Large Content Handling

• Inline markdown is capped at 20,000 characters (maxInlineContentChars).

• Stdio mode: full markdown is embedded as a resource block; if cache is enabled and content exceeds the inline limit, a resource_link is still included.

• HTTP mode: if content exceeds the inline limit and cache is enabled, the response includes a resource_link to superfetch://cache/... and omits embedded markdown. If cache is disabled, the inline markdown is truncated with ...[truncated].

• Upstream fetch size is capped at 10 MB of HTML; larger responses fail.

Resources

Resource listings enumerate cached entries, and subscriptions notify clients when cache entries update.

Download Endpoint (HTTP Mode)

When running in HTTP mode, cached content can be downloaded directly. Downloads are available only when cache is enabled.

Endpoint

• namespace: markdown

• Auth required (Authorization: Bearer <token>; in static token mode, X-API-Key is accepted)

Response Headers

Example Usage

Error Responses

Configuration

Set environment variables in your MCP client env or in the shell before starting the server.

Core Server Settings

For HTTP server tuning (SERVER_HEADERS_TIMEOUT_MS, SERVER_REQUEST_TIMEOUT_MS, SERVER_KEEP_ALIVE_TIMEOUT_MS, SERVER_SHUTDOWN_CLOSE_IDLE, SERVER_SHUTDOWN_CLOSE_ALL), see CONFIGURATION.md.

Auth (HTTP Mode)

Static mode requires at least one token (ACCESS_TOKENS or API_KEY).

OAuth (HTTP Mode)

Required when AUTH_MODE=oauth (or auto-selected by presence of OAuth URLs):

Optional:

Fixed Limits (Not Configurable via env)

• Fetch timeout: 15 seconds

• Max redirects: 5

• Max HTML response size: 10 MB

• Inline markdown limit: 20,000 characters

• Cache max entries: 100

• Session TTL: 30 minutes

• Session init timeout: 10 seconds

• Max sessions: 200

• Rate limit: 100 req/min per IP (60s window)

See CONFIGURATION.md for preset examples and quick-start snippets.

HTTP Mode Details

HTTP mode uses the MCP Streamable HTTP transport. The workflow is:

1. POST /mcp with an initialize request and no mcp-session-id header.

2. The server returns mcp-session-id in the response headers.

3. Use that header for subsequent POST /mcp, GET /mcp, and DELETE /mcp requests.

If the mcp-protocol-version header is missing, the server rejects the request. Only mcp-protocol-version: 2025-11-25 is supported.

GET /mcp and DELETE /mcp require mcp-session-id. POST /mcp without an initialize request will return 400.

Additional HTTP transport notes:

• POST /mcp should advertise Accept: application/json, text/event-stream (the server normalizes missing or */* Accept headers).

• GET /mcp requires Accept: text/event-stream (otherwise 406).

• JSON-RPC batch requests are not supported (400).

If the server reaches its session cap (200), it evicts the oldest session when possible; otherwise it returns a 503.

Host and Origin headers are always validated. Allowed values include loopback hosts, the configured HOST (if not a wildcard), and any entries in ALLOWED_HOSTS. When binding to 0.0.0.0 or ::, set ALLOWED_HOSTS to the hostnames clients will send.

Security

SSRF Protection

Blocked destinations include:

• Loopback and unspecified addresses (127.0.0.0/8, ::1, 0.0.0.0, ::)

• Private/ULA ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, fc00::/7)

• Link-local and shared address space (169.254.0.0/16, 100.64.0.0/10, fe80::/10)

• Multicast/reserved ranges (224.0.0.0/4, 240.0.0.0/4, ff00::/8)

• IPv6 transition ranges (64:ff9b::/96, 64:ff9b:1::/48, 2001::/32, 2002::/16)

• Cloud metadata endpoints (AWS/GCP/Azure/Alibaba) like 169.254.169.254, metadata.google.internal, metadata.azure.com, 100.100.100.200, instance-data

• Internal suffixes such as .local and .internal

DNS resolution is performed and blocked if any resolved IP matches a blocked range.

URL Validation

• Only http and https URLs

• No embedded credentials in URLs

• Max URL length: 2048 characters

• Hostnames ending in .local or .internal are rejected

Host/Origin Validation (HTTP Mode)

• Host header must match loopback, configured HOST (if not a wildcard), or ALLOWED_HOSTS

• Origin header (when present) is validated against the same allow-list

Rate Limiting

Rate limiting applies to /mcp and /mcp/downloads (100 req/min per IP, 60s window). OPTIONS requests are not rate-limited.

Development

Scripts

Note: Tests run via node --test with --experimental-transform-types to execute .ts test files. Node will emit an experimental warning.

Tech Stack

Contributing

1. Fork the repository

2. Create a feature branch: git checkout -b feature/amazing-feature

3. Ensure linting passes: npm run lint

4. Run tests: npm test

5. Commit changes: git commit -m 'Add amazing feature'

6. Push: git push origin feature/amazing-feature

7. Open a Pull Request

For examples of other MCP servers, see: github.com/modelcontextprotocol/servers