scrape-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| SCRAPE_MCP_PROXY | No | 代理，L1/L2 共用 | |
| SCRAPE_MCP_DATA_DIR | No | 登录态/缓存放盘目录 | ~/.scrape_mcp |
| SCRAPE_MCP_CACHE_TTL | No | L1 缓存 TTL（秒），`0` 关闭 | 300 |
| SCRAPE_MCP_HTTP_PORT | No | HTTP 端口 | |
| SCRAPE_MCP_TRANSPORT | No | 传输协议，stdio 或 streamable-http | stdio |
| SCRAPE_MCP_L2_CHANNEL | No | 关联浏览器：空=自带 Chromium，`chrome`/`msedge`=系统 | |
| SCRAPE_MCP_L2_ENABLED | No | 是否启用 Playwright 升级链 | true |
| SCRAPE_MCP_IMPERSONATE | No | curl_cffi 指纹模板 | chrome |
| SCRAPE_MCP_L2_TIMEZONE | No | 隐身加固时区 | Asia/Shanghai |
| SCRAPE_MCP_BATCH_MAX_URLS | No | `web_batch` 单批上限 | 20 |
| SCRAPE_MCP_REQUEST_TIMEOUT | No | L1 超时（秒） | 20 |
| SCRAPE_MCP_L2_SETTLE_TIMEOUT | No | 挑战页等待窗口（秒） | 8 |
| SCRAPE_MCP_BATCH_MAX_CONCURRENCY | No | `web_batch` 并发上限 | 4 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| web_fetchA | 抓取网页并返回极省 token 的紧凑正文（自动辨别反爬拦截页并如实上报）。 Args: url: 目标网页完整 URL，需带 http/https 协议头。 max_tokens: 正文 token 预算上限，超出时保留头尾、省略中段。 link_policy: 链接处理策略。internal=站内链接保留为相对路径、站外降级为纯文本（默认，最省）； all=站内外链接全保留；none=全部降级为纯文本。 |
| web_batchA | 并发抓取一批 URL，各自走完整分级链路，返回逐个结果。 Args: urls: 目标 URL 列表（最多 batch_max_urls 个，默认 20）。 max_tokens: 每个 URL 的正文 token 预算上限。 link_policy: 同 web_fetch。 |
| loginA | 打开带界面的浏览器手动登录，并把该站登录态持久化到磁盘。 用于 web_fetch 报告 login_required=true 的站点：浏览器会以非无头方式打开目标页， 请在弹出的窗口里完成登录；检测到登录成功或超时后，登录态（cookie 等）会被保存， 之后该站的 web_fetch 会自动带上登录态。 Args: url: 目标站点任一页面 URL（按域名区分登录态）。 timeout: 等待手动登录完成的秒数，超时也会保存当前状态便于下次继续。 |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 3 tools
web_fetch handles a single URL, web_batch handles multiple URLs concurrently, and login handles authentication persistence. These purposes are completely distinct, so an agent should never struggle to choose the right tool.
web_fetch and web_batch share a clear web_ prefix and read as fetch/batch operations, which is consistent. login deviates from the prefix pattern, but its purpose is obvious and it is still a simple verb-style name.
Three tools is a tight, focused set for a scraping server: single fetch, batch fetch, and authentication handling. There is no redundancy or bloat, and each tool earns its place.
The main scraping flows are covered: one-page fetches, multi-page fetches, and login-gated sites. Minor gaps exist, such as no way to explicitly manage or clear saved login sessions, but agents can still complete the core scraping lifecycle.