web_crawlerCrawl same-host HTML pages within configured depth and page limits to inventory links and forms for web application testing.
Crawl a bounded number of same-host HTML pages and inventory links and forms.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| max_depth | No | ||
| max_pages | No |
| Name | Required | Description | Default |
|---|---|---|---|
| pages | Yes | ||
| limits | Yes | ||
| page_count | Yes |
Changes observed during successful MCP inspections.
v2.2.0Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, and open-world, so the safety profile is covered. The description adds the meaningful behavioral constraints 'same-host' and 'bounded number', but says nothing about rate limiting, JS rendering, or what happens when depth/page bounds are hit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with the action and scope front-loaded. No filler, no redundant restatement of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists so return values needn't be explained, but with 0% parameter coverage and no usage guidance, an agent lacks enough to size the crawl correctly or know when to prefer it over crawling alternatives. The description is under-specified for a bounded, configurable crawler.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and none of the three parameters (url, max_depth, max_pages) are named or explained. 'Bounded number' vaguely gestures at max_pages, but max_depth and the url constraints are completely undocumented, so the description fails to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (crawl), a scoped resource (same-host HTML pages), and the output (inventory links and forms). This separates it from endpoint-discovery siblings like javascript_endpoint_discovery or gau_url_discovery, though it does not explicitly name any sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance or alternatives are given. The description never mentions when a crawl is preferable to sitemap_analysis, robots_txt_analysis, or gau_url_discovery, all of which it overlaps with. The 'same-host' phrase implies scope but gives no exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.