crawl_download
Save crawl results to one JSONL file with URL, status, and page content, returning only the file path and line count to keep responses small.
Instructions
Save every result of a crawl to one local file, as JSON lines: url, content_status and, once fetched, content (the page). Returns the file's path and line count, not the results, so a crawl of any size costs one small reply. Read the file with your own tools.
The file goes to path when given (a relative path inside ~/.zenrows/crawls; an absolute path or one that leaves that folder fails with INVALID_PATH), else ~/.zenrows/crawls/<crawl_id>.jsonl. An existing file is kept and the call fails with FILE_EXISTS unless overwrite is true.
On a running crawl the file holds only what was kept so far: download again once crawl_wait says it ended. After crawl_stop, pages in flight can still add lines for up to 10 minutes: the reply says so with partial: true. On the hosted server no file is written: the reply gives the download URL and a command to run where your API key is set.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| path | No | File to write, relative to ~/.zenrows/crawls (default <crawl_id>.jsonl) | |
| crawl_id | Yes | Crawl id returned by crawl_create (c_…) | |
| overwrite | No | Replace the file if it exists (default false) |