Yet Another WebFetch MCP Alternative

English | 中文

English

A local MCP server that fetches any http/https webpage, extracts the main content, and converts it to Markdown. Designed as a drop-in replacement for Claude Code's built-in WebFetch tool, which may fail under certain network / proxy environments (e.g., "Unable to verify if domain is safe").

Features

• Content extraction — Mozilla Readability for article pages; falls back to full-page conversion for non-article content.

• HTML → Markdown — powered by turndown; preserves headings and code blocks.

• Auto proxy detection — picks up Windows / macOS / Linux system proxy automatically (env vars also supported).

• Truncation — defaults to 50,000 characters to prevent context overflow.

• 20s timeout, redirect-follow, browser User-Agent.

• SSRF protection — http/https only; blocks localhost and private IP ranges (IPv4 & IPv6).

Installation

Requires Node.js ≥ 20.

git clone <repo-url> Yet-Another-WebFetch-MCP-Alternative
cd Yet-Another-WebFetch-MCP-Alternative
npm install

Setup with Claude Code

Add the following to ~/.claude.json under mcpServers (user-level) or to a project-local .mcp.json:

{
  "mcpServers": {
    "webfetch": {
      "type": "stdio",
      "command": "node",
      "args": ["<path-to-repo>\\index.js"]
    }
  }
}

Note: On Windows, avoid using npx as the command — it silently fails in non-TTY environments.

Reload the window (Ctrl+Shift+P → Developer: Reload Window), then type /mcp in a conversation to verify the connection.

Usage

The tool is registered as fetch_web.

Claude will invoke the tool automatically when it needs to fetch a webpage — no manual prompting required.

Proxy

The server auto-detects the system proxy on all three platforms:

To override manually, set the HTTPS_PROXY or HTTP_PROXY environment variable in the MCP config:

{
  "webfetch": {
    "type": "stdio",
    "command": "node",
    "args": ["<path-to-repo>\\index.js"],
    "env": {
      "HTTPS_PROXY": "http://127.0.0.1:7890"
    }
  }
}

Testing

npm test          # Handshake & tool list test (no network required)
LIVE=1 npm test   # End-to-end fetch against docs.tavily.com

Known Limitations

• No JavaScript rendering — dynamic pages only return the initial HTML (same as most scraping tools).

• Non-UTF-8 pages may produce garbled output in edge cases.

• Sites with strict anti-bot measures may return error pages or block requests.

• Domains unreachable from the local network (e.g., DNS-blocked in certain regions) cannot be fetched.

License

MIT

Related MCP server: superFetch MCP Server

中文

一款本地 MCP 服务器：抓取任意 http/https 网页，提取正文并转换为 Markdown。作为 Claude Code 内置 WebFetch 工具的替代——内置 WebFetch 在部分网络 / 代理环境下会报 "Unable to verify if domain is safe" 而不可用。

特性

• 正文提取 — 基于 Mozilla Readability，非文章页面自动回退全文转换。

• HTML → Markdown — 基于 turndown，保留标题与代码块。

• 自动代理检测 — 自动读取 Windows / macOS / Linux 系统代理，也支持环境变量覆盖。

• 截断 — 默认 50000 字符，防止上下文溢出。

• 20s 超时、重定向跟随、浏览器 UA。

• SSRF 防护 — 仅 http/https，拒绝本机与内网地址（含 IPv6）。

安装

需要 Node.js ≥ 20。

git clone <仓库地址> Yet-Another-WebFetch-MCP-Alternative
cd Yet-Another-WebFetch-MCP-Alternative
npm install

接入 Claude Code

在 ~/.claude.json 的 mcpServers（用户全局）或项目根目录 .mcp.json 中加入：

{
  "mcpServers": {
    "webfetch": {
      "type": "stdio",
      "command": "node",
      "args": ["<仓库路径>\\index.js"]
    }
  }
}

注意： Windows 下不要用 npx 作为 command（无 TTY 环境下会静默失败）。

保存后重载窗口（Ctrl+Shift+P → Developer: Reload Window），在对话中输入 /mcp 确认连接状态。

使用

工具名为 fetch_web。

Claude 会在需要抓取网页时自动调用，无需手动引导。

代理

服务器自动检测三平台系统代理：

如需手动覆盖，在 MCP 配置中设置 HTTPS_PROXY 或 HTTP_PROXY 环境变量：

{
  "webfetch": {
    "type": "stdio",
    "command": "node",
    "args": ["<仓库路径>\\index.js"],
    "env": {
      "HTTPS_PROXY": "http://127.0.0.1:7890"
    }
  }
}

测试

npm test          # 握手与工具清单测试（不依赖网络）
LIVE=1 npm test   # 额外做一次端到端抓取（docs.tavily.com）

已知限制

• 不渲染 JavaScript — 动态页面只能拿到初始 HTML（与多数抓取工具一致）。

• 非 UTF-8 编码页面在极端情况下可能乱码。

• 严格反爬站点可能被拦截或返回错误页。

• 本机网络 DNS 不可达的域名无法抓取。

许可证

MIT