Skip to main content
Glama

diogenes-web-scraping-engine-mcp

Web Hyperlink, Anchor & Asset Extraction Filter

extract_page_hyperlinks
Read-onlyIdempotent

Extracts, normalizes, and filters internal and external hyperlinks, anchor texts, and downloadable asset targets from HTML. Use when building site crawlers, auditing broken links, or extracting reference citations; for isolating primary readable article narrative text, use scrape_readable_article instead. Stateless read-only edge call (max 32 KB, 60 RPM): accepts 'html_content' ('link_scope' defaults to 'all_links'), supports '?demo=true' or L402 (10 sats), and returns HTTP 400 if invalid.

Input Schema

NameRequiredDescriptionDefault
base_urlNoBase URL of the target page for resolving relative link paths.
link_scopeNoLink extraction scope: all links, internal domain only, external links only, or download assets.
html_contentYesSource HTML string containing anchor tags, href attributes, and link targets.

Output Schema

NameRequiredDescriptionDefault
linksYesArray of link objects containing resolved url, anchor_text, and is_external.
total_linksYesTotal hyperlinks discovered matching the specified link scope.
unique_domainsYesList of distinct outbound domain hostnames linked in the document.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/idempotent/non-destructive, and the description goes well beyond them: stateless edge call, 32 KB payload ceiling, 60 RPM rate limit, '?demo=true' trial or L402 payment of 10 sats, and HTTP 400 on invalid input. These are operational facts an agent cannot get from anywhere else and that materially affect call planning.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences, front-loaded with purpose, then routing, then operational constraints. Nothing is wasted, though the pricing/payment clause is packed into the same sentence as the rate limit, making the tradeoff-heavy tail slightly harder to scan.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a stateless, read-only extractor with a full output schema, annotations covering safety, and 100% schema coverage, the description closes every remaining gap: cost model, rate limit, size ceiling, failure mode, and sibling routing. An agent has everything needed to decide and call correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3, but the description adds a value not present in the schema: link_scope defaults to 'all_links'. It also reinforces that html_content is the required input. It does not explain base_url resolution semantics beyond what the schema already says, so it stops short of a 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb triple (extracts, normalizes, filters) plus concrete resource types (hyperlinks, anchor texts, downloadable asset targets) read from HTML. It is immediately distinguishable from siblings like scrape_readable_article and sanitize_html_content because the description names the exact artifact produced.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It lists three concrete use cases (site crawlers, broken-link audits, citation extraction) and then explicitly routes the agent away to scrape_readable_article when the goal is article narrative text. That is an explicit when-to-use plus a named alternative, which is the top of the scale.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources