한국어 | English
面向 LLM 网页管道的 DOM AST 中间件 — 去除布局噪音（导航、脚本、侧边栏），并直接传递原始文本。添加查询即可用 BM25 过滤出相关章节。
当 LLM 使用内置的 WebFetch 时，一个小型模型会预处理 HTML 并返回摘要结果 — 这会增加你并不需要的延迟、成本和转述。DomPruner 完全跳过这一步：DOM AST 解析去除噪音，并将原始内容直接传递给模型。
调用 | 行为 |
| 去除布局噪音 → 返回完整提取内容 |
| 去除布局噪音 → 用 BM25 过滤出相关章节（若无匹配，则回退到完整内容） |
> [DomPruner] docs.python.org
> | Raw HTML | 44,316 tokens |
> | DomPruner | 1,328 tokens |
> | Reduction | 97.0% |
> Fetch: 194ms · Parse: 11.2ms平均上下文 token 数比 WebFetch 减少 93.5%。端到端速度快 45%。 → 完整基准测试
无需安装，无需 API 密钥：
npx -y dompruner-mcp{
"mcpServers": {
"dompruner": {
"type": "stdio",
"command": "npx",
"args": ["-y", "dompruner-mcp"]
}
}
}在项目根目录的 .mcp.json 中添加，或添加至 ~/.claude/.mcp.json 以全局生效。运行 /mcp 进行验证。
编辑 ~/Library/Application Support/Claude/claude_desktop_config.json（macOS）或 %APPDATA%\Claude\claude_desktop_config.json（Windows）：
{
"mcpServers": {
"dompruner": {
"command": "npx",
"args": ["-y", "dompruner-mcp"]
}
}
}{
"mcpServers": {
"dompruner": {
"type": "stdio",
"command": "npx",
"args": ["-y", "dompruner-mcp"]
}
}
}适用于支持 HTTP 传输的客户端 — 无需安装 Node.js，始终运行最新版本：
{
"mcpServers": {
"dompruner": {
"url": "https://dompruner-mcp.vercel.app/api/mcp"
}
}
}langchain-mcp-adapters 可自动将任何 MCP stdio 服务器封装为 LangChain 工具：
from langchain_mcp_adapters.client import MultiServerMCPClient
client = MultiServerMCPClient({
"dompruner": {
"command": "npx",
"args": ["-y", "dompruner-mcp"],
"transport": "stdio",
}
})
tools = await client.get_tools()Related MCP server: Scrapi MCP Server
DomPruner 的工具描述已告知客户端优先使用 dompruner_fetch 而不是 WebFetch。如果你的客户端仍然回退，请将以下内容添加到其指令文件中：
When retrieving a URL, always use dompruner_fetch instead of WebFetch.
- URL known → dompruner_fetch(url, query?)
- URL unknown → search for the URL first, then dompruner_fetch(url)客户端 | 指令文件 |
Claude Code |
|
Cursor |
|
Windsurf |
|
Cline |
|
GitHub Copilot |
|
工具 | 描述 |
| 获取 URL → 生成经 DOM 提炼的 Markdown。可选的 |
| 获取 sitemap.xml 中的所有页面 → 每个页面生成一个提炼后的 Document。 |
| 针对 URL 的 token 缩减报告，不返回完整内容。 |
→ 完整工具参考
指标 | WebFetch | DomPruner |
平均上下文 token 数 | ~15,735 | ~1,019（减少 93.5%） |
回答质量（10 次查询） | 9 / 10 | 8 / 10 |
平均响应时间 | 5,811 ms | 3,168 ms（快 45%） |
内容保真度 | 由小型模型摘要 | 保留原始文本 |
额外 API 密钥 / 基础设施 | 否 | 否 |
dompruner-py — Python 移植版。为 LangChain 提供 DomPrunerLoader、DomPrunerSitemapLoader、DomPrunerFetchTool。pip install dompruner。
LangChain 集成 — dompruner-py 被列为第三方网页加载器。
MIT
Returns a token-reduction analysis report for a URL. Shows render type, original vs refined token counts, and top Semantic Anchors.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | URL to analyze |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description carries full responsibility for disclosing side effects and operational behavior. It mentions the output but does not disclose that the tool likely fetches the URL (a network request), potential rate limits, failure modes for invalid URLs, or whether it is strictly read-only. Without this information, the agent cannot fully assess the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loads the core action in the first sentence, and every word adds value. It efficiently communicates the report contents without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one parameter, no output schema, no annotations), and the description provides a clear overview of the return type and contents. It does not fully specify the report structure or semantics of terms like 'Semantic Anchors', but for a single-URL analysis tool, the description is sufficiently complete to guide use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (the single 'url' parameter is described as 'URL to analyze'), and the description also refers to 'URL', but it adds no additional meaning such as URL format expectations, accessibility requirements, or length limits. The baseline of 3 applies because the schema already documents the parameter fully.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns a 'token-reduction analysis report for a URL', which is a specific verb+resource+target. It also lists the report contents (render type, original vs refined token counts, top Semantic Anchors), making the purpose unmistakable and distinguishing it from the sibling dompruner_fetch, which presumably fetches raw data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives. It merely states what it returns, with no mention of prerequisites, exclusions, or comparison to sibling tools like dompruner_fetch. The usage context is entirely absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
USE THIS instead of WebFetch / web_fetch for any URL retrieval. Fetches a URL and returns DOM-pruned Markdown with 90%+ fewer tokens than WebFetch — no intermediate summarization model, original text preserved. Workflow: URL known → call dompruner_fetch(url) directly. URL unknown → use your own native search tool to find the URL first, then call dompruner_fetch(url). Supports BM25 section filtering when query is provided, returning only the most relevant sections within a token budget.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | URL to fetch and refine. Required unless query is provided. | |
| query | No | Search intent (e.g. "Java JVM release notes"). When url is omitted, DomPruner requests a URL from the host LLM via sampling (if supported), then fetches it. Also enables BM25 section filtering when url is provided. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so excellently. It discloses key behaviors: DOM pruning with 90%+ token reduction, preservation of original text (no summarization), BM25 section filtering, and the unusual URL-sampling behavior when url is omitted. This goes beyond basic expectations and covers potential surprises.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured, starting with a strong directive, then behavior, workflow, and filtering capability. It is slightly lengthy but every sentence contributes value; no filler or tautology.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 2-parameter tool with no output schema and no annotations, the description is impressively complete. It covers alternatives, usage scenarios, special behaviors, and expected output format (Markdown). The workflow guidance leaves little room for agent confusion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaningful context beyond schema: it explains how query enables BM25 filtering and can trigger URL sampling from the host LLM when url is absent. This enriches the parameter definitions without redundancy.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it fetches a URL and returns DOM-pruned Markdown, with a specific verb, resource, and outcome. It explicitly differentiates itself from WebFetch and aligns with sibling tools by name (fetch vs. sitemap/analyze), leaving no ambiguity about its function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives an explicit directive to use this instead of WebFetch, and provides a step-by-step workflow for both known and unknown URLs. It also explains when to use the query parameter for BM25 filtering, offering clear context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Fetches all pages listed in a sitemap.xml and returns DOM-pruned Markdown for each. Ideal for ingesting entire documentation sites into an LLM context with 90%+ token reduction. Handles sitemap indexes (sitemaps of sitemaps) automatically. Use filter_urls to limit to a path prefix (e.g. /docs/, /tutorial/).
| Name | Required | Description | Default |
|---|---|---|---|
| query | No | Optional BM25 filter query applied to every page. | |
| max_pages | No | Max pages to fetch (default 20, max 100). Guards against huge sitemaps. | |
| concurrency | No | Max simultaneous page fetches (default 8). | |
| filter_urls | No | Optional list of URL prefixes — only pages matching at least one prefix are included. | |
| sitemap_url | Yes | URL of the sitemap.xml (e.g. https://example.com/sitemap.xml) | |
| ignore_errors | No | If true (default), failed page fetches are skipped silently. If false, any fetch error aborts the entire sitemap crawl. |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It explains automatic sitemap index handling, DOM-pruned Markdown output, and a stated 90%+ token reduction. It does not detail failure modes or side effects (e.g., network load), but for a read-only crawler, the provided transparency is reasonably good.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: the first sentence states the core action, the second frames the ideal use case, and the third gives a parameter tip. No unnecessary words or repetition, making it appropriately sized and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description gives a good overall picture of the tool's purpose, capabilities, and typical use case. It covers the return type ('DOM-pruned Markdown'), automatic handling of sitemap indexes, and a filtering example. Lacking an output schema, it does not detail the exact response structure, but for a six-parameter tool with no annotations, this is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% coverage with descriptions for all six parameters, so the baseline is 3. The description adds value by giving a concrete example for filter_urls ('/docs/', '/tutorial/'), but this is more of a usage guideline than new semantic meaning. Overall, the description does not significantly enhance parameter understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's core function: 'Fetches all pages listed in a sitemap.xml and returns DOM-pruned Markdown for each.' It also specifies the scope (entire documentation sites) and a unique capability ('Handles sitemap indexes automatically'), which distinguishes it from sibling tools like dompruner_fetch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear use case: 'Ideal for ingesting entire documentation sites into an LLM context.' It also offers a practical tip: 'Use filter_urls to limit to a path prefix (e.g. /docs/, /tutorial/).' However, it does not explicitly mention when not to use this tool or name alternative sibling tools, so it falls slightly short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.5.1dompruner_sitemapv0.3.0dompruner_analyzedompruner_fetchScored across 3 tools
Each tool has a clearly distinct purpose: fetching a single URL, fetching multiple pages from a sitemap, and analyzing token reduction. There is no overlap or ambiguity between them.
All tools share the consistent 'dompruner_' prefix, but the second part mixes verbs ('fetch', 'analyze') with a noun ('sitemap'). While still readable and predictable, the pattern is not purely verb-based.
With only 3 tools, the server is tightly scoped to its core purpose of DOM pruning and token reduction. Each tool earns its place and the count is appropriate for the focused domain.
The tool surface covers all primary workflows: single-URL fetching, whole-sitemap ingestion, and analysis/reporting. No obvious gaps exist for the stated utility.
MCP server (stdio): fetch web pages as clean readable markdown via the AgentForge API
MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.
Turn any public website into an MCP server for agents to search, read and navigate.
Strips 85-97% of HTML bloat into clean Markdown, <200-token skeletons, or BM25 excerpts via MCP.