Skip to main content
Glama
ZenRows
by ZenRows

crawl_download

Save crawl results to one JSONL file with URL, status, and page content, returning only the file path and line count to keep responses small.

Instructions

Save every result of a crawl to one local file, as JSON lines: url, content_status and, once fetched, content (the page). Returns the file's path and line count, not the results, so a crawl of any size costs one small reply. Read the file with your own tools.

The file goes to path when given (a relative path inside ~/.zenrows/crawls; an absolute path or one that leaves that folder fails with INVALID_PATH), else ~/.zenrows/crawls/<crawl_id>.jsonl. An existing file is kept and the call fails with FILE_EXISTS unless overwrite is true.

On a running crawl the file holds only what was kept so far: download again once crawl_wait says it ended. After crawl_stop, pages in flight can still add lines for up to 10 minutes: the reply says so with partial: true. On the hosted server no file is written: the reply gives the download URL and a command to run where your API key is set.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
pathNoFile to write, relative to ~/.zenrows/crawls (default <crawl_id>.jsonl)
crawl_idYesCrawl id returned by crawl_create (c_…)
overwriteNoReplace the file if it exists (default false)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Addedv2.7.0

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations: exact path resolution rules (~/.zenrows/crawls vs. absolute paths failing with INVALID_PATH), FILE_EXISTS behavior unless overwrite=true, partial:true semantics for in-flight pages after stop, and the hosted-server fallback returning a URL plus a command. This is rich, non-obvious operational detail an agent could not infer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense but front-loaded: the purpose and cheap-reply rationale come first, then path/overwrite rules, then lifecycle caveats. Every sentence carries information, though the three stacked paragraphs are at the upper edge of what's comfortable to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-value burden and does so explicitly ('Returns the file's path and line count'). Error cases, partial results, and the hosted-server variant round out a complete picture for a 3-parameter write tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: path is relative to ~/.zenrows/crawls, an absolute or escaping path fails with INVALID_PATH, and overwrite's default-false FILE_EXISTS interaction is spelled out. The crawl_id parameter is not elaborated but the schema already covers it (c_…).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Save every result of a crawl to one local file, as JSON lines' — and immediately clarifies the unusual return value ('the file's path and line count, not the results'). An agent can distinguish this from crawl_results/crawl_content without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains the context for re-invocation ('download again once crawl_wait says it ended'), the post-stop caveat with crawl_stop, and the hosted-server case where no file is written at all. What's missing is an explicit statement of when to prefer this over crawl_results; the contrast ('costs one small reply' vs. reading the file yourself) is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.