Skip to main content
Glama
PPParticle

Web Reader

by PPParticle
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

MCP Web Reader

一个自托管的 MCP (Model Context Protocol) 服务器，把网页内容读取并转换为 Markdown，供 Claude、Cursor 等 MCP 客户端使用。零第三方付费 API 依赖。

功能特点

  • 🔌 两个工具：read_url（单个）/ read_urls（批量，并发上限 3）

  • 🛤️ 专用通道：GitHub（raw README / Issue API）、arXiv（Atom API）、PDF 交接

  • 📄 统一提取：本地 fetch + Mozilla Readability + turndown，代码语言 / 表格 / 链接 / og: 元数据保真

  • 🌐 站点适配：微信公众号、知乎、掘金等按 host 优先选择器

  • 🧯 降级策略：本地解析 → 检测到访问限制（403 / 429 / Cloudflare 等）→ Playwright 浏览器渲染

  • ⚡ 缓存：文件系统缓存，按 host 分级 TTL；失败 URL 短期负缓存；自动淘汰

  • 🔁 重试：5xx / 429 / 网络错误指数退避重试

  • 🛡️ SSRF 防护：拦截私有 IP / 云元数据端点 / 危险 host

Related MCP server: crawl-mcp-server

安装

git clone https://github.com/PPParticle/web-reader.git
cd web-reader
npm install
npm run build
npx playwright install chromium   # 可选，用于 JS 渲染 / 反爬降级

配置

在任意 MCP 客户端的配置中添加（以 Claude Desktop 为例）：

Windows: %APPDATA%\Claude\claude_desktop_config.json macOS: ~/Library/Application Support/Claude/claude_desktop_config.json Linux: 自行查阅所用客户端的配置路径

{
  "mcpServers": {
    "web-reader": {
      "command": "node",
      "args": ["/absolute/path/to/web-reader/dist/index.js"]
    }
  }
}

环境变量（可选）：

变量

作用

GITHUB_TOKEN

提升 GitHub API 限额（60 → 5000/小时）；不设也能用公开仓库

WEB_READER_CACHE_DIR

覆盖缓存目录（默认 ~/.cache/mcp-web-reader/）

工具

read_url

读取单个 URL 并转为 Markdown。

参数

类型

默认

说明

url

string

（必填）

http/https URL

engine

enum

auto

auto=自动；local=强制本地解析；playwright=强制浏览器；jina=已移除（付费第三方，调用会返回明确错误）

no_cache

bool

false

为 true 时跳过缓存、重新抓取

with_links

bool

false

为 true 时在结果末尾附上正文链接清单，供 agent 决定是否进一步探索

read_urls

批量读取（最多 10 个 URL），参数同上（urls 数组 + engine + no_cache），单条失败不影响其他。

贡献

欢迎提交 Pull Request。

许可证

MIT License

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityMaintained
ResponsivenessSyncing

Resources

Related MCP Connectors

Related MCP Servers

  • Fetch MCP Server by modelcontextprotocol

    Fetch MCP Serverofficial

    modelcontextprotocol
    A
    license
    A
    quality
    A
    maintenance
    This server enables LLMs to retrieve and process content from web pages, converting HTML to markdown for easier consumption.
    1
    103,531 PyPI
    90,399
    MIT

  • crawl-mcp-server

    Git-Fg
    F
    license
    Not graded
    quality
    Not graded
    maintenance
    An MCP server for web content extraction that converts HTML pages into clean, LLM-optimized Markdown using Mozilla's Readability. It supports batch processing, intelligent multi-page crawling, and configurable caching while respecting robots.txt standards.
    33 npm
    -

  • superFetch MCP Server

    j0hanz
    A
    license
    A
    quality
    C
    maintenance
    An MCP server that fetches web pages and extracts clean, AI-friendly Markdown content using Mozilla Readability. It provides secure web access for LLMs with built-in SSRF protection and automated content cleaning for improved context retrieval and summarization.
    1
    42 npm
    MIT

  • markdown-for-agents-mcp

    JohnnyFoulds
    A
    license
    Not graded
    quality
    C
    maintenance
    MCP server for AI agents -- fetch any URL with full JavaScript rendering (Playwright/Chromium) and convert to clean, token-efficient markdown. Works on React, Vue, Angular, and any JS-heavy page. Includes web search, batch fetching, binary file download, LRU cache, SSRF protection, and structured output.
    16 npm
    MIT