Skip to main content
Glama
Llamatron2112

mcp-dom-extract

mcp-dom-extract

MCP server that downloads a web page, parses its DOM and extracts precise data via CSS selectors, then returns the structured result (JSON) to the client.

Requirements

  • Node.js ≥ 20.18.1

Related MCP server: Ashra Structured Data Extractor MCP

Installation and development

npm install
npm run build   # compiles src/ → dist/
npm run dev     # runs the server in watch mode (tsx)
npm run smoke   # end-to-end test against a local page

Exposed tools

extract_data_from_url

Downloads a page and extracts values via CSS selectors.

Parameter

Type

Description

url

string

Absolute URL of the page

selectors

array

List of { name, css, attribute?, multiple? }

max_value_length

number

Maximum length of each extracted value; longer values are truncated (default: 2000, cap: 100000)

  • attribute: if set (e.g. href, src), returns the attribute value instead of the text.

  • multiple: if true, returns an array of all matches.

extract_page_text

Downloads a page and returns its readable text, optionally scoped to a CSS selector and truncated to max_length characters.

Parameter

Default

Description

url

Absolute URL of the page

selector

CSS selector scoping the text; defaults to the whole document

max_length

20000

Maximum number of characters returned (cap: 100000)

inspect_page_structure

Returns a condensed DOM outline — tags, ids, classes, truncated text samples, repeated identical siblings aggregated (count + samples) — to pick relevant selectors without loading the full page into the context.

Parameter

Default

Description

max_nodes

200

Maximum number of outline entries (cap: 500)

max_text_length

40

Maximum length of each sample (cap: 200)

discover_selectors

The model describes what it is looking for with keywords; the server scans the DOM and returns candidate selectors whose text matches, with a sample. No full page enters the context.

{
  "url": "https://example.com/product",
  "targets": [
    { "name": "price", "keywords": ["19.99", "$", "price"] },
    { "name": "name", "keywords": ["Widget", "product"] }
  ]
}

Parameter

Default

Description

url

Absolute URL of the page

targets

List of { name, keywords }; keywords are matched case-insensitively against element text (1–10 targets, up to 10 keywords each)

max_candidates

5

Maximum number of candidate selectors returned per target (cap: 10)

probe_selectors

Checks selectors at low cost: for each, returns the number of matches and a sample of the first one, to refine before the final extraction.

Parameter

Type

Description

url

string

Absolute URL of the page

selectors

array

List of { name, css, attribute? } (multiple is not supported)

  1. discover_selectors (or inspect_page_structure) → candidate selectors

  2. probe_selectors → verification and refinement

  3. extract_data_from_url → final structured data

robots.txt compliance (RFC 9309)

Before each fetch, the server checks /robots.txt of the target origin and applies the rules of the mcp-dom-extract group (or * if no specific group exists). The cache is 24 h per origin, per the RFC.

  • 404 / 4xx errors → access allowed (the file is "unavailable").

  • 5xx / network / timeout errors → extraction blocked: the file is "unreachable", RFC 9309 requires a complete disallow. Unreachable states are re-checked after 5 minutes instead of being cached for 24 h.

  • Parsing limit of 512 KiB (the RFC requires at least 500 KiB).

  • crawl-delay is not supported (non-standard extension outside RFC 9309).

To disable the check (e.g. for internal test sites):

MCP_DOM_EXTRACT_IGNORE_ROBOTS=1 node dist/index.js

Page caching

Fetched pages are kept in an in-memory LRU cache, keyed by URL, so a multi-step workflow (discover_selectorsprobe_selectorsextract_data_from_url) fetches each page once instead of once per tool call. Cache entries expire after a TTL, the total size is bounded (oldest entries are evicted first), errors are never cached, and concurrent requests for the same URL share a single fetch.

Env var

Default

Description

MCP_DOM_EXTRACT_CACHE_TTL_MS

300000 (5 min)

How long a fetched page is reused before re-fetching

MCP_DOM_EXTRACT_CACHE_MAX_BYTES

52428800 (50 MiB)

Total cache size limit; set to 0 to disable caching

Client configuration

Claude Desktop standard (mcpServers)

This is the format used by Claude Desktop, Cherry Studio and most MCP clients (the emerging standard):

{
  "mcpServers": {
    "mcp-dom-extract": {
      "command": "npx",
      "args": ["-y", "--allow-git=all", "github:Llamatron2112/mcp-dom-extract"]
    }
  }
}

Zed uses the same entry under the mcp_servers key instead.

Distribution from GitHub

For npx github:user/mcp-dom-extract to work without a build step on the client side, the dist/ folder generated by npm run build must be committed. The bin field of package.json points to dist/index.js.

Since npm 11.10+ (and by default in npm 12), installing packages directly from Git is disabled for security reasons (EALLOWGIT). Pass --allow-git=all to opt in:

npx -y --allow-git=all github:Llamatron2112/mcp-dom-extract

Known limitations

  • JavaScript-rendered pages (SPAs): client-side generated content is not visible without a headless browser (Playwright would be a future addition).

  • The sites' terms of use remain your responsibility: the server applies robots.txt, not ToS.

  • Page encoding is detected by cheerio; pages with exotic charsets may be decoded incorrectly.

Available Tools

5 tools
discover_selectorsA

Fetches a web page and finds CSS selectors for elements whose text matches the given keywords. The model describes what it is looking for with keywords; the server returns candidate selectors with short samples, without the full page entering the context.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
targetsYesWhat to look for: a name plus keywords matched case-insensitively against element text
max_candidatesNoMaximum number of candidate selectors returned per target

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral burden. It discloses that the page is fetched, only candidate selectors with short samples are returned, and the full page does not enter the context. It does not cover failure modes or auth, but the key behavior is well explained.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences with no filler. The core function is front-loaded, and the behavioral caveat about context is placed second. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a three-parameter tool with no output schema, the description covers the inputs, the main behavior, and the output format (candidate selectors with short samples). It lacks only minor details like failure behavior, which are not essential for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions already cover targets and max_candidates (67% coverage), and url is self-explanatory. The tool description does not add parameter-specific meaning beyond what the schema already states, so it neither helps nor hurts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action: fetch a web page and find CSS selectors for elements whose text matches given keywords. It clearly distinguishes this from sibling tools like extract_page_text and probe_selectors by describing the matching mechanism and the output shape.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description conveys a clear use case: the model provides keywords and the server returns candidate selectors without loading the full page into context. This implies when to use it, but it does not explicitly name alternatives or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_data_from_urlB

Fetches a web page and extracts precise data from its DOM using CSS selectors.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
selectorsYes
max_value_lengthNoMaximum length of each extracted value; longer values are truncated

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the core fetch-and-extract behavior and does not mention limitations such as JavaScript rendering, selector failure handling, page-load behavior, rate limits, or whether the operation is safe and read-only.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, tight, front-loaded sentence with no wasted words. It communicates the core action and differentiator efficiently, though its brevity means it omits useful parameter and behavioral details that the low schema coverage cannot supply.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and three inputs, the description is under-specified for safely invoking this tool. An agent is left without information about the return shape, error behavior, or how this tool relates to the sibling tools, which is a notable gap for a network-fetching and DOM-extraction tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33%, and the main description adds little beyond the schema's structural requirements. It does not explain how selectors map to returned values, what 'name' represents, how 'multiple' changes results, or how 'max_value_length' truncates data, leaving the agent to infer this from the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Fetches a web page and extracts precise data'), a clear resource ('web page'), and a specific method ('using CSS selectors'). It differentiates from sibling tools like extract_page_text and inspect_page_structure by emphasizing precise, selector-based extraction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool: when precise, selector-based data extraction from a page's DOM is needed. However, it does not explicitly state when to prefer this over siblings like extract_page_text or discover_selectors, nor does it mention any exclusions or conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_page_textB

Fetches a web page and returns its readable text, optionally scoped to a CSS selector.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
selectorNoCSS selector scoping the text; defaults to the whole document
max_lengthNo

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden; it establishes that the tool performs a fetch and returns extracted text, suggesting a read operation. It does not disclose potential side effects like network calls and rate limits, or behavior such as truncation via max_length and errors on non-HTML pages, so transparency is only partially adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single tight sentence with the primary action front-loaded and the optional modifier at the end. There is no redundancy or filler; every word contributes to understanding the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 3-parameter read tool the description covers the core flow, but it omits what `max_length` actually limits and does not mention error behavior or output format. Since there is no output schema or annotations, these gaps make the definition only minimally complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 33%, and the tool description adds little beyond it: it echoes the selector scoping and frames `url` as the page to fetch, but leaves `max_length` semantically unexplained. The description does not compensate for the schema gaps for two of the three parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb ('Fetches') and resource ('a web page'), and names the output ('readable text') plus an optional CSS selector scope. It clearly points to extracting page text rather than structured data or DOM structure, though it does not explicitly reference sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The described behavior implies the main use case—reading text from a page, optionally focused via selector—but gives no explicit guidance on when to prefer this over extract_data_from_url, inspect_page_structure, discover_selectors, or probe_selectors. There are no when-not-to-use conditions or alternative routing, so the agent must infer placement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

inspect_page_structureA

Fetches a web page and returns a condensed structural outline of its DOM (tags, ids, classes, short text samples, repeated siblings aggregated). Lets the model pick relevant CSS selectors without loading the full page into context.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
max_nodesNoMaximum number of outline entries
max_text_lengthNoMaximum length of each text sample

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral disclosure burden. It explains the main behavior (fetching and condensing the DOM), including notable aggregation of repeated siblings. However, it does not disclose possible failure modes, network effects, rendering behavior, or limits on page size/complexity, leaving some gaps for an online operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that wastes no words. It states the action, the output, key structural features, and the practical purpose, making every phrase informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there is no output schema, the description reasonably explains what the tool returns and why it is useful. It would be stronger with a note about failure/error behavior or relationship to discover_selectors, but it otherwise gives an agent enough context to invoke the tool appropriately.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67%: max_nodes and max_text_length have descriptions, while url relies on its format type. The tool description does not explicitly clarify parameter usage, but it implies 'url' via 'Fetches a web page' and gives context for size limits through 'condensed' and 'short text samples'. This is adequate but not especially enriching beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Fetches'), resource ('web page'), and output ('condensed structural outline of its DOM'), and connects it to the distinct goal of picking CSS selectors. This clearly separates it from siblings like extract_page_text, which would focus on textual content rather than structure.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Lets the model pick relevant CSS selectors' gives a clear use context, and 'without loading the full page into context' provides a motivation for choosing it. It does not explicitly mention alternatives or when not to use it, but the context is sufficiently clear for an agent to make a reasonable choice among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

probe_selectorsA

Fetches a web page and reports, for each CSS selector, how many elements match and a short sample of the first match. Use it to verify or refine candidate selectors cheaply.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYes
selectorsYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It explains the basic behavior and output shape (match counts and a sample of the first match), but it does not disclose potential fetch failures, dynamic content limitations, or whether the operation is purely read-only. This is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with no wasted words. The primary action is front-loaded first, followed by a clear usage recommendation. It is appropriately sized for its purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter tool with no output schema, the description gives a reasonable overview and states return behavior. However, it lacks details about the structure of the 'selectors' items, possible error behaviors, and how this tool relates to the sibling tools. It is minimally complete but leaves gaps an agent must infer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains the general role of selectors, but it never defines the required 'name'/'css' fields or the optional 'attribute' field, which the schema leaves only minimally documented. This is a notable gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action ('Fetches a web page') and the resource ('for each CSS selector... how many elements match'), making the tool's core function easy to grasp. It does not explicitly contrast itself with sibling tools like discover_selectors or inspect_page_structure, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Use it to verify or refine candidate selectors cheaply' gives clear context for when the tool is appropriate. However, it does not mention when not to use it or name alternative tools, leaving some routing decision to the agent.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A3.8/5.0
Disambiguation4/5

Most tools have clear, distinct roles: structure inspection, selector discovery, selector probing, and extraction. The only slight overlap is between extract_data_from_url and extract_page_text, since both can retrieve content via CSS selectors, but the former emphasizes precise structured data while the latter focuses on readable text.

Naming Consistency5/5

All tool names follow a consistent snake_case verb_noun pattern: extract_data, extract_page, inspect_page, discover_selectors, probe_selectors. The 'from_url' suffix on one tool is a minor variation but does not break the overall predictability.

Tool Count5/5

Five tools is well-scoped for a DOM extraction server. Each tool represents a distinct step in the workflow of inspecting, discovering, probing, and extracting, with no redundancy or bloat.

Completeness5/5

The tool surface covers the full practical workflow for DOM extraction: inspect structure to understand the page, discover candidate selectors, probe selectors to verify them, then extract either precise data or readable text. There are no obvious dead ends or missing critical operations.

Maintenance

ActivityMaintained
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables extracting data from websites using natural language prompts, allowing users to specify exactly what content they want in plain English and returning structured JSON data.
    29
    8
    MIT
  • -
    license
    Not graded
    quality
    C
    maintenance
    Extract structured data from any website with a simple SDK call. No scraping code, no headless browsers - just prompt and get JSON.
    62
  • A
    license
    A
    quality
    C
    maintenance
    Enables browser-based web page fetching with automatic cookie management and CSS selector extraction, supporting JavaScript rendering and authenticated sessions through a Chrome extension.
    1
    7
    2
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Llamatron2112/mcp-dom-extract'

If you have feedback or need assistance with the MCP directory API, please join our Discord server