universal-scraper
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| US_POOL_SIZE | No | The size of the browser session pool used by the scraper. Default is 3. | 3 |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| scrapeA | 一键抓取一个 URL，返回干净文本/Markdown/正文/表格/链接。适合：读网页内容、取正文、找表格、收集链接。 |
| autoA | 用一句话描述爬虫任务，AI 自动生成配置、自动运行、失败自修复，返回结构化结果。适合：不会写选择器/配置的非技术用户。 |
| crawlA | 从入口 URL 递归爬取整站/子目录，返回每页标题+正文+链接。适合：整站抓取、文档站镜像。 |
| bulk_scrapeA | 批量抓取多个 URL（一次调用替代 N 次往返；逐 URL 隔离失败——单个 URL 出错不影响其余）。默认最多 10 个、上限 30（防爆）。每个 URL 有间隔（默认 1s）保持礼貌；返回每条的 url/status/摘要。 |
| screenshotA | 对页面截图（浏览器渲染）并保存为图片文件，返回文件路径。用于人工核对渲染结果/排障；图片本身不内联返回。 |
| extractA | 把 HTML 源码转成干净 Markdown/正文/表格，用于省 token 地阅读网页内容。默认先净化（剥离隐藏元素/注释/零宽字符等提示注入载体）再转换；main_only=true 时额外收窄到主内容容器（更适合阅读文章页）。 |
| booksA | 图书目录采集：按 ISBN 清单抓取豆瓣评分/作者/出版社、京东+当当聚合比价，导出 booklist.csv / booklist.md / books.json / crawl_log.md。只采集书目数据与公开封面，不下载电子书/PDF，不绕过登录或付费墙；0 条/部分失败会返回逐项诊断与行动方案（status=NO_DATA/PARTIAL），不会伪装成功。 |
| checkB | 引擎体检：返回版本、已通过测试（100+100+27）、能力矩阵、LLM 可用性、目录结构。 |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 8 tools
Most tools have distinct roles (crawl, bulk_scrape, screenshot, check, books), but scrape and extract overlap heavily because both turn web content into clean Markdown/text/tables, differing mainly by URL vs HTML input. The auto tool also overlaps conceptually with scrape/crawl by generating and running scraping tasks, so agents may hesitate between auto, scrape, and extract.
Names mix single verbs (scrape, crawl, check, extract), a noun-like tool (books), a vague automation label (auto), and one snake_case compound (bulk_scrape). The set is readable but has no predictable verb_noun or resource_action convention.
Eight tools is well-scoped for a universal scraper, covering single-page scraping, batch scraping, recursive crawling, extraction, screenshots, diagnostics, AI-assisted automation, and a book-data specialization. Each tool has a plausible place in the workflow.
The surface covers the core scraping lifecycle: fetch one page, fetch many, crawl a site, extract from HTML, screenshot, diagnose, and automate configuration. Minor gaps exist around explicit session/cookie/proxy management or structured job status, but the main workflows are covered.