Skip to main content
Glama

web_scrape

Fetch a public web page and return its content as clean markdown (default), with optional html, raw_html, links, and screenshot. Strips nav/footer/sidebar by default so you get the main article body.

It reads pages that are not otherwise locally accessible — docs, blog posts, marketing pages, articles. The response is page metadata (title, description, language) plus the requested formats. If the page served a bot-challenge interstitial ("verify you are human") instead of real content, the response carries challenge_detected: true — that content is not the page, and an immediate retry usually returns the same challenge.

Screenshot format (formats: ["screenshot"]) uses the same three-tier answer as browser: small enough (auto-downscaled) → inline data:image/...;base64,...; too large with no storage → a short-TTL scratch_url you don't have to clean up; storage: "/path.jpg" set → written permanently to project files, returned as { storage_path, size_bytes }.

Example:

{ "project_id": "my-saas", "url": "https://somewhere.cloud/docs/getting-started.txt" }
// Returns: { "url": "...", "title": "Start an app", "markdown": "# Start an app\n...", "status_code": 200 }
{ "project_id": "my-saas", "url": "https://example.com", "formats": ["markdown", "links"], "only_main": false }
{ "project_id": "my-saas", "url": "https://example.com", "formats": ["screenshot"], "storage": "/scratch/hero.jpg" }

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "project_id": "<project_id>",
      "url": "<url>"
    },
    "name": "web_scrape"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYeshttp(s) URL to scrape, max 2048 chars.
formatsNoOptional array (or JSON-stringified array) of formats to return. Allowed: markdown, html, rawHtml, links, screenshot. Default: ["markdown"]. Example: '["markdown","links"]'.
storageNoOnly meaningful with formats:["screenshot"]. A project files path like "/scratch/hero.jpg" to write the shot permanently instead of the default inline/ephemeral behavior. Returns { storage_path, size_bytes } instead of the image bytes.
wait_forNoOptional milliseconds to wait for the page to settle before scraping (max 30000). For SPAs that hydrate after load.
only_mainNoStrip nav/footer/sidebar to keep only the main article content. Default true.
project_idYesProject ID (UUID), subdomain, or slug — resolved server-side, no UUID lookup needed. 'default' also works when the account has exactly one project; multi-project accounts must name one (the error lists them).

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds substantial behavior beyond the annotations: the default main-content stripping, the challenge_detected flag with the fact that retrying usually fails, auto-downscaling of screenshots, short-TTL scratch URLs, and permanent writes when `storage` is set. These details also explain why readOnlyHint=false is plausible, given that `storage` writes to project files. A minor gap remains around rate limits and error behavior for failed fetches.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The definition is front-loaded: the purpose and default output come before the reference material. The bolded 'Screenshot format' block and the three JSON examples keep the dense screenshot behavior scannable. It runs long for a scraper, but nearly every sentence carries operational detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must carry the return contract, and it does: page metadata (title, description, language), requested format fields, the challenge_detected flag, and the alternate storage return shape. An agent can predict the response shape for every documented call pattern.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description goes further. It explains the three-tier screenshot delivery, what `storage` returns ({ storage_path, size_bytes }) instead of image bytes, and what the default `formats` value and `only_main` stripping mean. Worked examples show how the parameters combine.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names a specific verb and resource plus the exact output formats: 'Fetch a public web page and return its content as clean markdown (default), with optional html, raw_html, links, and screenshot.' It also states a default transformation (stripping nav/footer/sidebar). It references the `browser` sibling only to explain the screenshot delivery tiers, not to distinguish when each tool should be used, so sibling differentiation is partial.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives usage context ('pages that are not otherwise locally accessible — docs, blog posts, marketing pages, articles') but never states when NOT to use it or which sibling to pick instead. With `browser`, `docs`, and `web_search` in the catalog, the agent must infer the boundary itself, for example whether to use this tool for interactive or JavaScript-heavy pages.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources