Skip to main content
Glama
Sign In
scrapy-mcp

scrapy-mcp

by eitan3
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

crawl_results

Retrieve scraped items from a running or finished crawl with cursor-based pagination, using a limit per page and continuing with the returned next cursor.

Instructions

Page through a crawl's scraped items.

cursor: line offset to start from (use the returned next_cursor to continue). limit: items per page (1-1000). Safe to call while the crawl is still running -- you get whatever has been written so far.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "job_id": "<job_id>"
    },
    "name": "crawl_results"
  }
}

Input Schema

NameRequiredDescriptionDefault
limitNo
cursorNo
job_idYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses key behavioral traits: cursor is a line offset, next_cursor is used to continue, limit range (1-1000), and that results reflect whatever has been written so far during an active crawl. This adds valuable context beyond the schema and clarifies the tool's eventual-consistency behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is brief and front-loads the main purpose. It then uses a clear list-style block for parameter guidance. Every sentence adds value, with no filler or repetition. The formatting is clean and readable.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple pagination tool with three parameters and no output schema, the description covers the primary purpose, parameter semantics, and a key behavioral nuance. It hints at the response shape via 'next_cursor' but does not explicitly describe the structure of items. Overall, this is complete enough for effective tool invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description compensates well by explaining cursor and limit in detail. It does not explicitly describe job_id, but the tool name and 'a crawl's' context make it obvious that job_id identifies the crawl. This is sufficient for a required identifier parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb "Page through" and clearly identifies the resource as "a crawl's scraped items." This distinguishes it from sibling tools like start_crawl or crawl_status, making its function unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use the tool (to retrieve crawl results in pages) and explains the pagination mechanism via cursor and limit. It also notes it can be safely used while the crawl is running, which implies appropriate usage during execution. It does not explicitly exclude alternatives, but the context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools