Skip to main content
Glama

Web Page to RAG Chunks

page_rag
Read-only

3 free MCP calls per day (all tools together), then $0.0025 USDC via x402 or card credits (paid in-band: params._meta["x402/payment"]). Page RAG: Any public web page -> clean LLM-ready Markdown + RAG chunks in one call: main content via Mozilla Readability (no nav/ads/footers), absolute links, title/description/language/canonical/published+modified dates/JSON-LD types, word + token estimates, sha256 content hash for change detection, and heading-aware chunks (configurable size + overlap, each with its heading path). Plain HTTP, robots.txt respected, Markdown/plain text passed through. Flags pages that likely need JavaScript. When to use: Use page-rag to feed a web page into an LLM or a vector store. Use page-meta or page-check for SEO/meta tags and HTTP status only. page-markdown is the older, simpler converter; page-rag is recommended.

Input Schema

NameRequiredDescriptionDefault
urlYespublic http(s) page URL
overlapNochunk overlap in tokens (default 80)
main_onlyNomain content only via Readability (default true)
max_charsNomax Markdown characters (default 100000, max 400000)
chunk_tokensNochunk size in tokens (0 = no chunks, default 800, max 4000)

Output Schema

NameRequiredDescriptionDefault
okNo
urlNo
titleNo
chunksNo
statusNo
fetchMsNo
sourcesNo
finalUrlNo
languageNo
markdownNo
siteNameNo
canonicalNo
extractorNo
fetchedAtNo
truncatedNo
wordCountNo
modifiedAtNo
contentHashNo
contentTypeNo
descriptionNo
jsonLdTypesNo
publishedAtNo
tokenEstimateNo
truncatedReasonNo
likelyNeedsJavaScriptNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedOutput schema / description
      Previous value: -"Inferred from a real response captured on 2026-10-24; extra fields may appear, null-valued fields are untyped."New value: +"Inferred from a real response captured on 2026-10-26; extra fields may appear, null-valued fields are untyped."
  2. Changed1 schema field changed
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "description": "Inferred from a real response captured on 2026-10-24; extra fields may appear, null-valued fields are untyped.",
      +  "properties": {
      +    "canonical": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "chunks": {
      +      "items": {
      +        "properties": {
      +          "headingPath": {
      +            "type": [
      +              "string",
      +              "null"
      +            ]
      +          },
      +          "index": {
      +            "type": [
      +              "number",
      +              "null"
      +            ]
      +          },
      +          "text": {
      +            "type": [
      +              "string",
      +              "null"
      +            ]
      +          },
      +          "tokenEstimate": {
      +            "type": [
      +              "number",
      +              "null"
      +            ]
      +          }
      +        },
      +        "type": "object"
      +      },
      +      "type": "array"
      +    },
      +    "contentHash": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "contentType": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "description": {},
      +    "extractor": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "fetchMs": {
      +      "type": [
      +        "number",
      +        "null"
      +      ]
      +    },
      +    "fetchedAt": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "finalUrl": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "jsonLdTypes": {
      +      "type": [
      +        "array",
      +        "null"
      +      ]
      +    },
      +    "language": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "likelyNeedsJavaScript": {
      +      "type": [
      +        "boolean",
      +        "null"
      +      ]
      +    },
      +    "markdown": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "modifiedAt": {},
      +    "ok": {
      +      "type": [
      +        "boolean",
      +        "null"
      +      ]
      +    },
      +    "publishedAt": {},
      +    "siteName": {},
      +    "sources": {
      +      "items": {
      +        "properties": {
      +          "name": {
      +            "type": [
      +              "string",
      +              "null"
      +            ]
      +          },
      +          "url": {
      +            "type": [
      +              "string",
      +              "null"
      +            ]
      +          }
      +        },
      +        "type": "object"
      +      },
      +      "type": "array"
      +    },
      +    "status": {
      +      "type": [
      +        "number",
      +        "null"
      +      ]
      +    },
      +    "title": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "tokenEstimate": {
      +      "type": [
      +        "number",
      +        "null"
      +      ]
      +    },
      +    "truncated": {
      +      "type": [
      +        "boolean",
      +        "null"
      +      ]
      +    },
      +    "truncatedReason": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "url": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "wordCount": {
      +      "type": [
      +        "number",
      +        "null"
      +      ]
      +    }
      +  },
      +  "type": "object"
      +}
  3. Added

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/openWorld annotations: discloses rate limits and pricing mechanics (3 free calls/day across all tools, $0.0025 USDC via x402 or card, paid in-band via params._meta), that fetching is plain HTTP with robots.txt respected, that Markdown/plain text pass through, and that JS-dependent pages are flagged. These are operational facts an agent cannot get from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense paragraph with no filler sentences — pricing, capability, when-to-use, and sibling routing all earn their place. It is slightly front-loaded with the billing preamble before the purpose statement, which delays the core 'what it does' by one clause, but nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers extraction behavior, chunking, limits and payment for a five-parameter open-world fetch tool, and an output schema exists so return structure need not be re-explained. Minor gap: no mention of failure modes (non-HTML responses, blocked/paywalled pages) or what a JS-flagged result looks like, which matters for an open-world scraper.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3; the description earns an increment by explaining what the chunk parameters actually produce ('heading-aware chunks (configurable size + overlap, each with its heading path)') and what main_only toggles ('main content via Mozilla Readability (no nav/ads/footers)'). It does not restate defaults or the 0 = no chunks sentinel, which remain schema-only.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb+resource: 'Any public web page -> clean LLM-ready Markdown + RAG chunks in one call', and enumerates the extracted artefacts (main content, metadata, hash, heading-aware chunks). It explicitly differentiates itself from page-markdown, page-meta and page-check, so an agent can pick it without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The 'When to use' block names the use case (feed a page into an LLM or vector store) and the alternatives with their conditions ('page-meta or page-check for SEO/meta tags and HTTP status only'; 'page-markdown is the older, simpler converter; page-rag is recommended'). Routing is fully determined by the description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources