Skip to main content
Glama

Read URL

scrape_url
Read-only

Reads one public web page by URL and returns its content as markdown (text, truncated, total_chars). Use it when the URL is known; it does not search (use google_search), and Zillow or eBay URLs are refused free (use search_zillow or search_ebay). Costs 1 credit, 5 with render_js, 10 with premium or challenge_mode, 25 with render_js and premium; blocked or empty pages are not charged. A page not ready within 25 seconds returns a job_id instead; async=true returns one at once. Poll with the same url and job_id; jobs are kept 1 hour. Required: url.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "scrape_url"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesPublic http or https page URL, at most 8192 characters, e.g. "https://example.com/pricing". Private network addresses and sign-in pages are refused; Reddit URLs are read from the public feed.
asyncNotrue returns {job_id, status, retry_after} at once instead of waiting; false (default) waits up to 25 seconds and returns a job_id only if the page is not ready by then. Boolean, e.g. true.
job_idNojob_id from an earlier scrape_url call: job_ followed by 64 lowercase hex characters. Send it with the same url to read the result; polling starts no new scrape and costs nothing. Jobs are kept 1 hour. Omit to start a new scrape.
premiumNoFetch through residential proxies, for pages that come back blocked or empty. Boolean, default false; 10 credits, 25 with render_js.
max_charsNoMaximum markdown characters returned in text, integer 1000-30000, default 12000, e.g. 5000. Longer pages are cut and truncated is true; total_chars gives the full length. Ignored when max_tokens is set.
render_jsNoRun the page's JavaScript in a browser before reading it, for pages built in the browser. Boolean, default false; 5 credits instead of 1.
max_tokensNoToken limit, integer 1-1000000, estimated at about 4 characters per token, e.g. 2000; replaces max_chars. Over it, text is cut and content_id/content_url fetch the full page for 1 hour with the same API key, at no extra charge. No default.
challenge_modeNoSeparate browser mode for pages that answer with a browser verification check. Boolean, default false; 10 credits. Cannot be combined with premium or render_js.
idempotency_keyNoCaller-chosen key, 1-128 characters, e.g. "order-42-pricing". A repeat call with the same key and options returns the same job instead of scraping and charging again; the same key with other options is rejected. Omit to start a new job on every call.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed9 schema fields changed
    • changedInput schema / properties / async / description
      Previous value: -"Return a pending job immediately; otherwise wait up to 25 seconds."New value: +"true returns {job_id, status, retry_after} at once instead of waiting; false (default) waits up to 25 seconds and returns a job_id only if the page is not ready by then. Boolean, e.g. true."
    • changedInput schema / properties / challenge_mode / description
      Previous value: -"For pages that answer with a browser verification check. 10 credits; not combinable with premium or render_js."New value: +"Separate browser mode for pages that answer with a browser verification check. Boolean, default false; 10 credits. Cannot be combined with premium or render_js."
    • changedInput schema / properties / idempotency_key / description
      Previous value: -"Reuse to avoid duplicate jobs and charges."New value: +"Caller-chosen key, 1-128 characters, e.g. \"order-42-pricing\". A repeat call with the same key and options returns the same job instead of scraping and charging again; the same key with other options is rejected. Omit to start a new job on every call."
    • changedInput schema / properties / job_id / description
      Previous value: -"Poll a previous scrape job. Keep the original url; no new scrape or charge."New value: +"job_id from an earlier scrape_url call: job_ followed by 64 lowercase hex characters. Send it with the same url to read the result; polling starts no new scrape and costs nothing. Jobs are kept 1 hour. Omit to start a new scrape."
    • changedInput schema / properties / max_chars / description
      Previous value: -"Maximum number of markdown characters to return."New value: +"Maximum markdown characters returned in text, integer 1000-30000, default 12000, e.g. 5000. Longer pages are cut and truncated is true; total_chars gives the full length. Ignored when max_tokens is set."
    • changedInput schema / properties / max_tokens / description
      Previous value: -"Optional token limit (~4 characters per token); replaces max_chars. Over it, text is cut and content_id/content_url fetch the full page for 1 hour with the same key, at no extra charge."New value: +"Token limit, integer 1-1000000, estimated at about 4 characters per token, e.g. 2000; replaces max_chars. Over it, text is cut and content_id/content_url fetch the full page for 1 hour with the same API key, at no extra charge. No default."
    • changedInput schema / properties / premium / description
      Previous value: -"Premium mode (10 credits): retry with this when a page comes back blocked or empty."New value: +"Fetch through residential proxies, for pages that come back blocked or empty. Boolean, default false; 10 credits, 25 with render_js."
    • changedInput schema / properties / render_js / description
      Previous value: -"Whether to render the page with JavaScript before scraping."New value: +"Run the page's JavaScript in a browser before reading it, for pages built in the browser. Boolean, default false; 5 credits instead of 1."
    • changedInput schema / properties / url / description
      Previous value: -"Public HTTP or HTTPS page URL to scrape."New value: +"Public http or https page URL, at most 8192 characters, e.g. \"https://example.com/pricing\". Private network addresses and sign-in pages are refused; Reddit URLs are read from the public feed."
  2. Changed3 schema fields changed
    • addedInput schema / properties / async
      Added value: +{
      +  "default": false,
      +  "description": "Return a pending job immediately; otherwise wait up to 25 seconds.",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / idempotency_key
      Added value: +{
      +  "description": "Reuse to avoid duplicate jobs and charges.",
      +  "maxLength": 128,
      +  "minLength": 1,
      +  "type": "string"
      +}
    • addedInput schema / properties / job_id
      Added value: +{
      +  "description": "Poll a previous scrape job. Keep the original url; no new scrape or charge.",
      +  "pattern": "^job_[a-f0-9]{64}$",
      +  "type": "string"
      +}
  3. Changed1 schema field changed
    • addedInput schema / properties / max_tokens
      Added value: +{
      +  "description": "Optional token limit (~4 characters per token); replaces max_chars. Over it, text is cut and content_id/content_url fetch the full page for 1 hour with the same key, at no extra charge.",
      +  "maximum": 1000000,
      +  "minimum": 1,
      +  "type": "integer"
      +}
  4. Added

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond annotations, it discloses credit costs including combinations, that blocked or empty pages are not charged, the 25-second timeout behavior, job retention (1 hour), and that polling with the same url and job_id costs nothing. The readOnly/openWorld annotations are consistent, and the description adds significant operational context that annotations cannot express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded with the core operation, then alternatives, then cost/job behavior. Every sentence carries actionable information; even the pricing sentence earns its place because costs affect tool choice and parameter combinations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter tool with no output schema, the description covers the main return fields, truncation behavior, job_id polling, timeout handling, cost implications, domain refusals, and key prerequisites. Nothing essential for correct invocation is omitted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline would be 3. The description still adds cross-parameter semantics not obvious from individual fields: that async=true returns a job_id immediately, that polling requires the same url plus job_id, that job_id polling starts no new scrape and costs nothing, and that render_js/premium/challenge_mode interact with pricing and cannot be combined.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise verb and resource ('Reads one public web page by URL') and discloses the return shape ('markdown (text, truncated, total_chars)'). It explicitly distinguishes itself from search-oriented siblings by naming google_search and refusing Zillow/eBay URLs in favor of search_zillow/search_ebay.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit use condition ('when the URL is known'), exclusions ('it does not search'), and named alternatives for unhandled domains ('use google_search', 'use search_zillow or search_ebay'). It also explains the async and job_id polling workflow so agents know when to use those parameters instead of waiting.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources