The superFetch MCP Server fetches public web pages and converts HTML into clean, AI-readable Markdown — ideal for feeding web content to LLMs.
Fetch & Convert Web Content: Retrieve content from any public HTTP(S) URL and transform it into clean Markdown, with noise removal (navbars, footers, clutter) applied by default
Code Host Integration: Automatically rewrite GitHub, GitLab, Bitbucket, and Gist URLs to raw content endpoints
Caching: Results are cached for reuse; bypass with
forceRefreshfor fresh content, or access cached content viainternal://cache/{namespace}/{hash}Inline Size Control & Task Mode: Limit returned Markdown size with
maxInlineChars; large or slow fetches can run asynchronously via task mode to avoid truncation/timeoutsStructured Metadata: Responses include
url,resolvedUrl,finalUrl,title,markdown,fromCache,fetchedAt,contentSize, andtruncatedBuilt-in Resources & Help: Access guidance via
internal://instructionsand aget-helpprompt for usage detailsHTTP Server Mode: Optional HTTP server with host/origin validation, authentication (static tokens, OAuth), rate limiting, and health checks
Security: SSRF protections, local/private network blocking, TLS support, and protocol version validation
Read-only: Purely a content fetcher — no JavaScript execution or page interaction
Broad Client Support: Works with VS Code, Cursor, Claude Desktop, LM Studio, and other MCP-compatible clients
Automatically rewrites Bitbucket URLs to their raw content versions to extract clean, readable text from code repositories.
Automatically rewrites GitHub and Gist URLs to their raw content versions to extract clean, readable text from code repositories.
Automatically rewrites GitLab URLs to their raw content versions to extract clean, readable text from code repositories.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@superFetch MCP Serverfetch and clean the content from https://www.wired.com/story/ai-chip-wars/"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Fetch URL MCP Server
An MCP server that fetches web pages and converts them to clean, readable Markdown.
Overview
This server takes a URL, fetches the page, and strips away everything you don't need — navigation, sidebars, banners, scripts — leaving just the main content as Markdown. It's perfect for feeding into LLMs, giving them the distilled essence of a page without the noise. It also recognizes GitHub, GitLab, Bitbucket, and Gist URLs and rewrites them to fetch the raw content directly.
By default it runs over stdio. Pass --http if you need a proper HTTP endpoint with auth, rate limiting, TLS, and session support.
Key Features
HTML to Markdown — Turns any public web page into clean, readable Markdown with metadata like
title,url,contentSize, andtruncated.Smart URL handling — Recognizes GitHub, GitLab, Bitbucket, and Gist page URLs and rewrites them to raw-content endpoints before fetching.
Task mode — Big or slow pages can run as async MCP tasks with progress updates, instead of blocking. In HTTP mode, tasks are bound to the authenticated caller rather than a single MCP session, so they can be resumed after reconnecting with the same credentials. Polling task state also exposes
progressandtotalwhen available.Self-documenting — Includes an
internal://instructionsresource and aget-helpprompt so clients know how to use it.HTTP mode — Optionally serves over Streamable HTTP with host/origin validation, bearer or OAuth auth, rate limiting, health checks, and TLS.
Web Client
A browser-based client is available if you want to use the server without any MCP setup.
Requirements
Node.js >= 24
Docker (optional) — only needed if you want to run the container image
Quick Start
Add this to your MCP client config:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}Client Configuration
Add to .vscode/mcp.json:
{
"servers": {
"fetch-url-mcp": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}Or install via CLI:
code --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'For more info, see VS Code MCP docs.
Add to .vscode/mcp.json:
{
"servers": {
"fetch-url-mcp": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}Or install via CLI:
code-insiders --add-mcp '{"name":"fetch-url-mcp","command":"npx","args":["-y","@j0hanz/fetch-url-mcp@latest"]}'For more info, see VS Code Insiders MCP docs.
Add to ~/.cursor/mcp.json:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Cursor MCP docs.
For solution-scoped setup, add this to .mcp.json at the solution root:
{
"servers": {
"fetch-url-mcp": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Visual Studio MCP docs.
Add to ~/.config/goose/config.yaml on macOS/Linux or %APPDATA%\Block\goose\config\config.yaml on Windows:
extensions:
fetch-url-mcp:
name: fetch-url-mcp
cmd: npx
args: ['-y', '@j0hanz/fetch-url-mcp@latest']
enabled: true
type: stdio
timeout: 300For more info, see Goose extension docs.
Add to ~/.lmstudio/mcp.json on macOS/Linux or %USERPROFILE%/.lmstudio/mcp.json on Windows:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see LM Studio MCP docs.
Add to claude_desktop_config.json:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Claude Desktop MCP docs.
Use the CLI:
claude mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latestFor project-scoped config, Claude Code writes .mcp.json with:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
"env": {}
}
}
}For more info, see Claude Code MCP docs.
Add to ~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Windsurf MCP docs.
Add to ~/.config/amp/settings.json on macOS/Linux, %USERPROFILE%\.config\amp\settings.json on Windows, or .amp/settings.json for workspace-scoped config:
{
"amp.mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}Or install via CLI:
amp mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latestFor more info, see Amp docs.
Open the MCP Servers panel, choose Configure MCP Servers, and add this to cline_mcp_settings.json:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Cline MCP docs.
Use the CLI:
codex mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latestOr add this to ~/.codex/config.toml or project-scoped .codex/config.toml:
[mcp_servers.fetch-url-mcp]
command = "npx"
args = ["-y", "@j0hanz/fetch-url-mcp@latest"]For more info, see Codex MCP docs.
Add to .vscode/mcp.json:
{
"servers": {
"fetch-url-mcp": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see GitHub Copilot MCP docs.
Open Personal > MCP Servers in Warp, choose + Add, and either add a CLI server with:
command:npxargs:["-y", "@j0hanz/fetch-url-mcp@latest"]
Or paste this JSON snippet when using Warp's multi-server import flow:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Warp MCP docs.
Use Kiro's MCP Servers panel or the Add to Kiro install flow. Kiro stores workspace-scoped MCP config in .kiro/settings/mcp.json and user-scoped config in ~/.kiro/settings/mcp.json.
For this server, use:
command:npxargs:["-y", "@j0hanz/fetch-url-mcp@latest"]
For more info, see Kiro MCP docs.
Add to ~/.gemini/settings.json:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Gemini CLI MCP docs.
Add to ~/.config/zed/settings.json:
{
"context_servers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"],
"env": {}
}
}
}For more info, see Zed MCP docs.
Use the Augment Settings panel and either add the server manually or choose Import from JSON:
{
"mcpServers": {
"fetch-url-mcp": {
"command": "npx",
"args": ["-y", "@j0hanz/fetch-url-mcp@latest"]
}
}
}For more info, see Augment MCP docs.
Use Roo Code's MCP Servers UI or marketplace flow.
For this server, use:
command:npxargs:["-y", "@j0hanz/fetch-url-mcp@latest"]
For more info, see Roo Code docs.
Use Kilo Code's MCP Servers UI or marketplace flow.
For this server, use:
command:npxargs:["-y", "@j0hanz/fetch-url-mcp@latest"]
For more info, see Kilo Code docs.
Use Cases
Documentation for LLMs — Grab a docs page, blog post, or reference article as Markdown and pass it straight into a context window.
Repository content — Hand it a GitHub, GitLab, or Bitbucket URL and it resolves the raw content endpoint. Works with Gists too.
Slow or large pages — Task mode lets big fetches run in the background while sending monotonic progress updates back to the client, while
tasks/getexposes the lateststatusMessage,progress, andtotal.
Architecture
[MCP Client]
├─ stdio -> `src/index.ts` -> `startStdioServer()` -> `createMcpServer()`
└─ HTTP (`--http`) -> `src/index.ts` -> `startHttpServer()` -> HTTP dispatcher
├─ `GET /health`
├─ `GET /.well-known/oauth-protected-resource`
├─ `GET /.well-known/oauth-protected-resource/mcp`
└─ `POST|GET|DELETE /mcp`
`createMcpServer()`
├─ registers tool: `fetch-url`
├─ registers prompt: `get-help`
├─ registers resources:
│ - `internal://instructions`
├─ enables capabilities: logging, resources, prompts, tasks
└─ installs task handlers, log-level handling, and shutdown cleanup
`fetch-url` execution
├─ validate input with `fetchUrlInputSchema`
├─ normalize URL and block local/private targets unless allowed
├─ rewrite supported code-host URLs to raw endpoints when possible
├─ fetch content via the shared pipeline
├─ transform HTML into Markdown in the transform worker path
└─ validate `structuredContent` with `fetchUrlOutputSchema`Request Lifecycle
[Client] -- initialize {protocolVersion, capabilities} --> [Server]
[Server] -- {protocolVersion, capabilities, serverInfo} --> [Client]
[Client] -- notifications/initialized --> [Server]
[Client] -- tools/call {name, arguments} --> [Server]
[Server] -- {content: [{type, text}], structuredContent?, isError?} --> [Client]MCP Surface
Tools
fetch-url
Takes a URL and returns Markdown. Read-only — no JavaScript execution. Supports running as a background MCP task for large or slow pages. When task mode is used, tasks/get and tasks/list include statusMessage, progress, and total whenever progress has been reported.
Parameter | Type | Required | Description |
|
| yes | Target URL. Max 2048 chars. |
You get text content back by default. If output validation passes, the response also includes structuredContent with typed fields: url, resolvedUrl, finalUrl, title, metadata, markdown, fetchedAt, contentSize, and truncated. A true value for truncated means the content hit a server-side size limit.
To opt into progress updates, include _meta.progressToken in the tool call. The token may be a string or number. The server may then emit monotonic notifications/progress updates, and task mode reuses the same token until the task reaches a terminal state.
{
"method": "tools/call",
"params": {
"name": "fetch-url",
"arguments": {
"url": "https://example.com/docs"
},
"_meta": {
"progressToken": 7
}
}
}1. [Client] -- tools/call {name: "fetch-url", arguments} --> [Server]
2. [Server] -- dispatch("fetch-url") --> [src/tools/fetch-url.ts]
3. [Handler] -- validate(fetchUrlInputSchema) --> normalize / fetch / transform
4. [Handler] -- validate(fetchUrlOutputSchema) --> assemble content + structuredContent
5. [Server] -- result or tool error --> [Client]Resources
Resource | URI | MIME Type | Description |
|
|
| Guidance for using the Fetch URL MCP server. |
Prompts
Prompt | Arguments | Description |
| none | Return Fetch URL server instructions: workflows, task mode, and error handling. |
MCP Capabilities
Capability | Status | Notes |
completions | confirmed | Advertised in |
logging | confirmed | Advertised in |
resources subscribe/listChanged | confirmed | Advertised in |
prompts | confirmed |
|
tasks | confirmed | Advertised in |
progress notifications | confirmed | Opt-in via |
Tool Annotations
Annotation | Value |
|
|
|
|
|
|
|
|
Structured Output
The tool declares an outputSchema and includes structuredContent in the response when validation passes. Clients that support structured output get typed data directly; the rest use the text fallback.
Configuration
All configuration is through environment variables. For basic stdio usage, nothing needs to be set.
HTTP Server
Variable | Default | Notes |
|
| Bind address. Non-loopback bindings also require |
|
| Listening port for |
|
| Must be enabled to bind to a non-loopback interface. |
| empty | Additional allowed |
|
| Optional connection cap. |
| unset | Optional Node server tuning. |
| unset | Optional Node server tuning. |
| unset | Optional keep-alive tuning. |
| unset | Optional keep-alive tuning buffer. |
| unset | Optional header count limit. |
|
| Enables inbound private-network protections. |
Authentication & OAuth
Variable | Default | Notes |
| unset | Comma- or space-separated static bearer tokens. |
| unset | Alternate static token source for header auth. |
| unset | Enables OAuth mode when combined with the other OAuth URLs. |
| unset | Optional explicit authorization endpoint. |
| unset | Optional explicit token endpoint. |
| unset | Optional OAuth revocation endpoint. |
| unset | Optional OAuth dynamic client registration endpoint. |
| unset | Required for OAuth token introspection. |
| empty | Required scopes enforced after auth. |
| unset | Optional introspection client ID. |
| unset | Optional introspection client secret. |
TLS
Variable | Default | Notes |
| unset | Enable HTTPS when set together with |
| unset | TLS certificate path. |
| unset | Optional custom CA bundle. |
Fetching
Variable | Default | Notes |
|
| Allows loopback and private-network fetch targets. |
|
| Network fetch timeout in milliseconds. |
|
| Override the outbound user agent string. |
Tool Output
Variable | Default | Notes |
|
|
|
Tasks
Variable | Default | Notes |
|
| Total retained task capacity, including completed/cancelled tasks until they expire. |
|
| Per-owner retained task cap, clamped to the total cap. |
|
| Enables status notifications for tasks. |
|
| Requires interception for task-capable tool execution. |
Transform Workers
Variable | Default | Notes |
|
| Cancellation acknowledgement timeout. |
|
| Worker execution mode. |
| unset | Optional worker memory limit. |
| unset | Optional worker memory limit. |
| unset | Optional worker memory limit. |
| unset | Optional worker stack size. |
Content Cleanup
Variable | Default | Notes |
| empty | Extra noise-removal tokens. |
| empty | Extra DOM selectors for noise removal. |
| system default | Locale override for extraction heuristics. |
| built-in list | Override heading keywords used by cleanup. |
Logging
Variable | Default | Notes |
|
|
|
|
| Set to |
HTTP Endpoints
Method | Path | Auth | Purpose |
|
| no, unless | Basic health response, with optional diagnostics. |
|
| no | OAuth protected-resource metadata. |
|
| no | OAuth protected-resource metadata for the MCP endpoint. |
|
| yes | Session initialization and JSON-RPC requests. |
|
| yes | Session-bound server-to-client stream handling. |
|
| yes | Session shutdown. |
Security
Control | Status | Notes |
Host and origin validation | implemented | HTTP requests are rejected unless |
Authentication | implemented | HTTP mode supports static bearer tokens locally or OAuth token introspection; remote bindings require OAuth. |
Protocol version checks | implemented | Session-bound MCP HTTP requests validate |
Rate limiting | implemented | Requests pass through the HTTP rate limiter before route dispatch. |
Outbound SSRF protections | implemented | Local/private IPs, metadata endpoints, and |
TLS | optional | HTTPS is enabled when both TLS key and certificate files are configured. |
Stdio logging safety | implemented | Server logs are written to stderr, not stdout, so stdio MCP traffic stays clean. |
Development
Essential Commands
Command | Description |
| Clean, compile TypeScript, copy assets. |
| Watch mode TypeScript compilation. |
| Run the server with |
| Start the compiled server. |
| Run the full test suite. |
| Lint with ESLint. |
| Auto-fix lint issues. |
| Type-check source and tests. |
| Format with Prettier. |
| Build and launch MCP Inspector. |
Script | Command |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Build and Release
npm run prepublishOnlyruns lint, type-check, and build as a single release gate.CI workflows are under
.github/workflows/.Dockerfileanddocker-compose.ymlare included for containerized runs.Published on npm as
@j0hanz/fetch-url-mcp.
Troubleshooting
Symptom | Likely Cause | Fix |
Server output mixes with MCP traffic on stdio | Logs going to stdout | Ensure all logging writes to stderr; the server does this by default. |
HTTP mode returns | Host/origin mismatch | Add the domain to |
HTTP mode returns | Missing or invalid token | Set |
Fetch returns private-IP error | SSRF protections blocked the target | Set |
| Content exceeded inline limits | Increase |
Transform timeout or worker crash | Large or complex HTML | Tune |
Client config not working | Wrong config format for the client | Check the matching |
Credits
Dependency | Registry |
npm | |
npm | |
npm | |
npm | |
npm | |
npm |
Contributing and License
Pull requests welcome. Please make sure these pass before submitting:
npm run lintandnpm run type-checknpm testnpm run format
License
MIT License. See LICENSE for details.