Skip to main content
Glama

fetchWebContent

Read-onlyIdempotent

Fetch readable web pages, docs, and PDFs from public URLs, stripping ads while preserving code and tables. Scope by CSS selector, page long text, or extract structured JSON for research.

Instructions

Fetch readable web content/docs/PDFs from public URLs. Auto-cleans ads, keeps code/tables. Batch urls (1-3), CSS selector scoping, bm25 focus, startIndex paging, cache, schema extract to JSON, citations, link retention.

Input Schema

NameRequiredDescriptionDefault
rawNoReturn the raw response body (HTML/plain text) without extraction
urlNo
urlsNoBatch URLs (1~3 URLs run concurrently, splitting maxChars budget; ideal for comparing search results)
cacheNoCache mode: 'default' uses disk Markdown cache; 'reload' forces re-fetching upstream
focusNoKeyword or section name to target (e.g. 'Auth', 'Config'); returns targeted window (~5000 chars) instead of burning tokens on full document
formatNoContent format (default: 'markdown' when raw is false): 'markdown' preserves headings, lists, fenced code blocks (with language) and GFM tables — better for technical docs; use 'text' for plain text
extractNoExtract structured JSON instead of markdown, using CSS selectors — no LLM needed. fields: [{name, selector, type: text|html|markdown|attribute|list, attribute?, fields?, all?}]; selector '@attr' reads an attribute of the current node
maxCharsNo
selectorNoCSS selector: return only the matched subtree, keeping code blocks and tables. Use when auto-extraction grabs the wrong container. Errors instead of silently returning the whole page when nothing matches
citationsNoRewrite inline links as numbered citations and append a '## References' list, so the answer can cite sources
focusModeNoHow focus selects text: 'bm25' (default) ranks paragraphs by relevance and keeps the best ones; 'window' keeps the old ±2500-char window around the first match
startIndexNoCharacter offset to start reading from (for paging through long content)
readabilityNo
retainLinksNoInline link retention: 'all' (default) keeps [label](url); 'text' keeps only the label (strips URLs, big win on index/list pages); 'none' removes the link entirely. Ignored inside code blocks
includeLinksNo

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed4 schema fields changedv1.5.1
    • changedInput schema / additionalProperties
      Previous value: -trueNew value: +{}
    • changedInput schema / properties / extract / additionalProperties
      Previous value: -trueNew value: +{}
    • changedInput schema / properties / extract / properties / fields / items / additionalProperties
      Previous value: -trueNew value: +{}
    • addedInput schema / properties / startIndex / maximum
      Added value: +9007199254740991
  2. Changed7 schema fields changedv1.3.10
    • changedInput schema / additionalProperties
      Previous value: -falseNew value: +true
    • addedInput schema / properties / cache
      Added value: +{
      +  "description": "Cache mode: 'default' uses disk Markdown cache; 'reload' forces re-fetching upstream",
      +  "enum": [
      +    "default",
      +    "reload"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / citations
      Added value: +{
      +  "description": "Rewrite inline links as numbered citations and append a '## References' list, so the answer can cite sources",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / extract
      Added value: +{
      +  "additionalProperties": true,
      +  "description": "Extract structured JSON instead of markdown, using CSS selectors — no LLM needed. fields: [{name, selector, type: text|html|markdown|attribute|list, attribute?, fields?, all?}]; selector '@attr' reads an attribute of the current node",
      +  "properties": {
      +    "baseSelector": {
      +      "minLength": 1,
      +      "type": "string"
      +    },
      +    "fields": {
      +      "items": {
      +        "additionalProperties": true,
      +        "properties": {
      +          "all": {
      +            "type": "boolean"
      +          },
      +          "attribute": {
      +            "minLength": 1,
      +            "type": "string"
      +          },
      +          "fields": {
      +            "items": {},
      +            "type": "array"
      +          },
      +          "name": {
      +            "minLength": 1,
      +            "type": "string"
      +          },
      +          "selector": {
      +            "minLength": 1,
      +            "type": "string"
      +          },
      +          "type": {
      +            "enum": [
      +              "text",
      +              "html",
      +              "markdown",
      +              "attribute",
      +              "list"
      +            ],
      +            "type": "string"
      +          }
      +        },
      +        "required": [
      +          "name"
      +        ],
      +        "type": "object"
      +      },
      +      "maxItems": 50,
      +      "minItems": 1,
      +      "type": "array"
      +    }
      +  },
      +  "required": [
      +    "fields"
      +  ],
      +  "type": "object"
      +}
    • addedInput schema / properties / focusMode
      Added value: +{
      +  "description": "How focus selects text: 'bm25' (default) ranks paragraphs by relevance and keeps the best ones; 'window' keeps the old ±2500-char window around the first match",
      +  "enum": [
      +    "bm25",
      +    "window"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / retainLinks
      Added value: +{
      +  "description": "Inline link retention: 'all' (default) keeps [label](url); 'text' keeps only the label (strips URLs, big win on index/list pages); 'none' removes the link entirely. Ignored inside code blocks",
      +  "enum": [
      +    "all",
      +    "none",
      +    "text"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / selector
      Added value: +{
      +  "description": "CSS selector: return only the matched subtree, keeping code blocks and tables. Use when auto-extraction grabs the wrong container. Errors instead of silently returning the whole page when nothing matches",
      +  "minLength": 1,
      +  "type": "string"
      +}
  3. Changed3 schema fields changedv1.2.1
    • addedInput schema / properties / focus
      Added value: +{
      +  "description": "Keyword or section name to target (e.g. 'Auth', 'Config'); returns targeted window (~5000 chars) instead of burning tokens on full document",
      +  "minLength": 1,
      +  "type": "string"
      +}
    • addedInput schema / properties / urls
      Added value: +{
      +  "description": "Batch URLs (1~3 URLs run concurrently, splitting maxChars budget; ideal for comparing search results)",
      +  "items": {
      +    "format": "uri",
      +    "type": "string"
      +  },
      +  "maxItems": 3,
      +  "minItems": 1,
      +  "type": "array"
      +}
    • removedInput schema / required
      Removed value: -[
      -  "url"
      -]
  4. Changed1 schema field changedv1.0.25
    • changedInput schema / properties / format / description
      Previous value: -"Content format: 'markdown' preserves fenced code blocks (with language) and GFM tables — better for technical docs"New value: +"Content format (default: 'markdown' when raw is false): 'markdown' preserves headings, lists, fenced code blocks (with language) and GFM tables — better for technical docs; use 'text' for plain text"
  5. Changed1 schema field changedv1.0.15
    • addedInput schema / properties / format
      Added value: +{
      +  "description": "Content format: 'markdown' preserves fenced code blocks (with language) and GFM tables — better for technical docs",
      +  "enum": [
      +    "text",
      +    "markdown"
      +  ],
      +  "type": "string"
      +}
  6. First observedv1.0.11

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (read-only, idempotent, open-world, non-destructive), and the description still adds real behavior: ad stripping, disk Markdown caching, batch URLs splitting the maxChars budget, and selector failing loudly rather than silently returning the whole page. Missing are failure modes for inaccessible URLs, rate limiting, or what the default response looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The opening sentence is well front-loaded and earns its place. The follow-on feature list is telegraphic ('bm25 focus, startIndex paging, cache, schema extract to JSON') and largely restates schema parameters, adding tokens without adding meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 15-parameter tool with a nested extract object and no output schema, the description covers the main modes but omits that at least one of url/urls must be supplied (zero required params), and never characterizes the default return shape. It is minimally sufficient but leaves real gaps for a tool this complex.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 73%, so most parameters are already documented in the schema itself. The description's second sentence names capabilities (bm25 focus, startIndex paging, cache, citations, link retention) but adds no syntax or defaults beyond what the parameter descriptions already state, so it earns only the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetch readable web content/docs/PDFs from public URLs') plus the transformation applied ('Auto-cleans ads, keeps code/tables'). The 'public URLs' scope implicitly separates it from fetchGithubContent and queryDocs, but siblings are never named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Several in-line cues imply usage: batch urls are 'ideal for comparing search results', selector is for 'when auto-extraction grabs the wrong container', focus avoids 'burning tokens on full document'. However there is no explicit when-to-use versus fetchGithubContent/queryDocs/search, and no stated preconditions such as needing at least one of url or urls.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.