scrapewright

crawl_site

Walk a site and extract every item. Waits up to four minutes; a longer crawl returns a job_id to pass to crawl_status.

    `mode` picks how the items are found. "rows" (the default) reads every
    card on the listing itself and follows its pagination -- right for
    search results, categories and tables. "links" follows each card into
    its own page, for sites where an item has one. "like" starts from one
    example item page, given as `like_page`, and finds every other page
    shaped like it; pass `listing_url` as well when you know the stock or
    results page, because inferring it is the part that fails.

Input Schema

TableJSON Schema

Name	Required	Default
`js`	No
`mode`	No	rows
`fields`	No
`scroll`	No
`like_page`	No
`max_items`	No
`listing_url`	Yes

Output Schema

TableJSON Schema

Name	Required	Description	Default
No arguments

Schema Changelog

Changes observed during successful MCP inspections.

Changed2 schema fields changedSep 30, 2026

addedInput schema / properties / like_page

Added value: +{
+  "anyOf": [
+    {
+      "type": "string"
+    },
+    {
+      "type": "null"
+    }
+  ],
+  "default": null,
+  "title": "Like Page"
+}

addedInput schema / properties / mode

Added value: +{
+  "default": "rows",
+  "title": "Mode",
+  "type": "string"
+}

First observedSep 20, 2026

TDQS

A3.9/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does meaningful work: it discloses the four-minute timeout and the asynchronous job_id handoff to crawl_status, plus the failure-prone nature of inferring the listing URL in 'like' mode. It does not cover permission/auth needs or the behavior of js, scroll, and max_items, keeping it below a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The timeout/job_id behavior is front-loaded, followed by tightly-scoped per-mode guidance. Every sentence earns its place, though the mode prose is a dense run-on that could be structured more scannably.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described. The description covers the crawl flow, async handoff, and the mode semantics that matter most for calling it correctly; the gaps around js/scroll/max_items are the main shortfall for a 7-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 7 parameters, so the description must compensate. It explains `mode`, `like_page`, and `listing_url` well, but says nothing about `js`, `fields`, `scroll`, or `max_items`, leaving half the surface undocumented anywhere.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a clear verb+resource: 'Walk a site and extract every item.' It also names crawl_status as the follow-up tool for long crawls, giving an agent a way to distinguish this from the status poller. It does not differentiate itself from extract_page or detect_site, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly maps each `mode` value to a concrete situation ('rows' for search results/categories/tables, 'links' when items have own pages, 'like' for shape-matching from a sample page), and even notes when to also pass `listing_url`. That is strong when-to-use guidance for the parameter, though there is no guidance on choosing crawl_site over the sibling tools extract_page or detect_site.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All scrapewright tools Inspect

Try in Browser

Let your AI agent see the webWeb-to-screenshot MCP for Claude, Cursor and Codex. No cookie banners. Free to start.snap-render.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.