Skip to main content
Glama

CrawlerData: Google Shopping & SERP

Crawl a web page

crawl_page
Read-only

Fetch any public web page and return its content as markdown (default), HTML or text. Gets past common bot walls by switching to a real browser only when needed. Credits by what it took: 1 plain fetch, 3 with JavaScript rendering, 10 for hard-protected sites; nothing when the page could not be fetched.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "crawl_page"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesThe page to fetch (http or https).
formatNomarkdown (default), html or text.
wait_forNoCSS selector to wait for before returning the page.
render_jsNoRender JavaScript in a browser (3 credits). Off by default; the crawler switches to a browser by itself when the page needs one.
country_codeNo2-letter country to fetch from (default us).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint and openWorldHint; the description goes well beyond by disclosing the credit cost model (1 / 3 / 10) and, notably, that no credits are charged on failure. It also explains the bot-wall escalation behavior and that it is automatic, which is real operational context an agent cannot get from the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no filler, and the most decision-relevant facts come first: what it returns, then how it handles bot walls, then cost. Nothing repeats the input schema verbatim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only fetch tool with no output schema, the description covers return formats and the full cost/behavior profile. Minor gaps remain: no mention of content size limits, truncation, or how redirects/timeouts are handled, but nothing essential to calling it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter is already documented, including format, render_js, wait_for and country_code. The description adds little parameter-level meaning — the 1/3/10 credit ladder extends the schema's "3 credits" note slightly, but country_code and wait_for receive no treatment in the description. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

"Fetch any public web page and return its content as markdown (default), HTML or text" gives a specific verb (fetch/crawl), a specific resource (an arbitrary web page), and the three output forms. That is plainly distinguishable from the sibling google_* search/autocomplete/shopping tools, which are query tools rather than page retrievers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Any public web page" scopes usage to publicly reachable URLs, and the note that the crawler escalates to a browser "only when needed" tells the agent it does not need to pre-schedule rendering. There is no explicit when-not guidance (paywalls, auth-gated pages, or when to prefer a search sibling are unaddressed), which keeps it below a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources