Skip to main content
Glama

diogenes-web-scraping-engine-mcp

DOM Paywall, Modal Gate & Blurred Content Recovery Auditor

detect_paywall_and_bypass
Read-onlyIdempotent

Audits web page DOM hierarchies to detect soft paywalls, blur overlays, modal gates, and recovers concealed body text. Use when diagnosing paywall obstructions, diagnosing subscriber gating scripts, or auditing content access barriers; for assessing general SEO hygiene, use evaluate_seo_diagnostics instead. Stateless read-only edge call (max 32 KB, 60 RPM): accepts 'html_content' ('detection_profile' defaults to 'heuristic_dom_overlay'), supports '?demo=true' or L402 (15 sats), and returns HTTP 400 if invalid.

Input Schema

NameRequiredDescriptionDefault
html_contentYesTarget web page HTML markup suspected of containing soft paywalls or paywall modals.
target_domainNoOptional publishing domain name to look up publisher gating patterns.
detection_profileNoPaywall detection heuristic: heuristic DOM overlay, hidden text recovery, or strict access barrier audit.

Output Schema

NameRequiredDescriptionDefault
paywall_typeYesClassification of paywall barrier (e.g., soft_modal_overlay, blurred_content, hard_truncation).
confidence_scoreYesConfidence metric between 0.0 and 1.0 regarding paywall detection.
paywall_detectedYesTrue if paywall scripts, blur filters, or gate overlays were identified.
recovered_contentYesRecovered readable prose extracted from under DOM overlay or script tags.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnly/idempotent/non-destructive/openWorld=false; the description goes well beyond them by disclosing statelessness, the 32 KB payload cap, the 60 RPM rate limit, the demo mode and L402 pricing (15 sats), and the HTTP 400 failure mode for invalid input. Those are operational constraints an agent cannot infer from any structured field.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and the sibling alternative are front-loaded, then operational facts are packed into a single dense sentence. Nothing is redundant, but the rate-limit/payment/error clause is compressed into a run-on that costs some scanability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists so return shape need not be explained, and the description still covers the inputs, the default profile, the size and rate ceilings, the payment path, and the failure status. For a read-only three-parameter auditor, an agent has everything needed to invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds a default the schema omits ('detection_profile' defaults to 'heuristic_dom_overlay') and reiterates the html_content input and demo flag. It does not explain target_domain's publisher-pattern lookup, leaving one property to the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — audit DOM hierarchies for soft paywalls, blur overlays, modal gates, plus concealed body text recovery. The title's phrasing is echoed but the description adds concrete detection targets (blur overlays, modal gates) rather than restating the name, and it distinguishes the tool from evaluate_seo_diagnostics by naming that sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives both when-to-use (diagnosing paywall obstructions, subscriber gating scripts, content access barriers) and when-not-to-use with the exact alternative: 'for assessing general SEO hygiene, use evaluate_seo_diagnostics instead.' That is an explicit routing rule an agent can act on without reading either schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources