Skip to main content
Glama

Web Scrape (URL to Markdown)

web_scrape
Read-only

Web page to markdown: read any web page (URL) as clean, LLM-ready Markdown, the web scraper for agents. Default tool for reading ONE web page whose URL you have: returns clean Markdown (title, meta description, main content with headings, lists and links); Markdown and plain-text files such as llms.txt or README.md come back unchanged. The fastest and cheapest way to give an LLM the content of a page. If a JavaScript app comes back nearly empty, set render:true (real browser, slower, plain text) or use webpage-reader. For 2 to 5 URLs use web-read-batch, for 6 to 20 web-scrape-batch (one payment), for a whole site site-crawl, and if you have a question but no URL search-read. Params: url (required), max_chars (default 8000, max 60000), include_links, render. Price: $0.0015 USDC per call, paid inside this MCP call via x402 (USDC on Base, Solana, Polygon or Arbitrum). x402 MCP clients such as @x402/mcp pay automatically. Free sample: call with the example input {"url":"https://example.com","max_chars":8000}.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "web_scrape"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesPublic http(s) URL to scrape
renderNoRender with a headless browser (for JavaScript-heavy pages). Returns plain text instead of Markdown.
max_charsNoTruncate Markdown after this many characters
include_linksNoAlso return up to 100 absolute links found on the page

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare read-only/open-world/no-destruction, but the description adds substantial behavior beyond that: the $0.0015 USDC per-call price paid in-band via x402 on four chains, the automatic payment path for x402 MCP clients, the fact that Markdown/plain-text files like llms.txt pass through unchanged, and that render:true silently switches output to plain text and is slower. These are exactly the non-obvious traits an agent needs before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The structure is well front-loaded: capability, default scope, return shape, then routing, then params and pricing. It is dense but mostly earns its length; the free-sample call and the example input are marginally promotional and the param list partly repeats the schema, which keeps it from a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-URL fetch tool with no output schema, the description covers everything material: what comes back, how truncation and links work, the JS-rendering escape hatch, the payment mechanism and cost, and which sibling handles other cardinalities. An agent can select and invoke this correctly without further inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and each parameter is already documented, so the baseline is 3. The description still adds value by restating the operative defaults and ceiling (max_chars default 8000, max 60000) and by explaining the output-format consequence of render, which the schema only hints at. It doesn't add much on include_links or URL validity rules beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('read any web page (URL) as clean, LLM-ready Markdown') and specifies what is returned: title, meta description, main content with headings, lists and links. It also carves out a distinct niche against siblings by declaring itself the 'Default tool for reading ONE web page', which separates it from web_scrape_batch and web-read-batch.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit routing rules: this tool for exactly one URL, web-read-batch for 2–5 URLs, web-scrape-batch for 6–20 (with the 'one payment' note), site-crawl for a whole site, and search-read when there's a question but no URL. It also prescribes the fallback path when a JS app returns nearly empty (set render:true or use webpage-reader), so both the when and the when-not are covered.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources