batch_crawl
Crawl up to 3 URLs concurrently with automatic fallback on failure. Optionally save each page as markdown plus an index.json to a local directory.
Instructions
Crawl multiple URLs with fallback. Max 3 URLs per call. Use output_path (directory) to persist full per-URL markdown + index.json; the return shape stays a list, each success item gets an output_file key.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| urls | Yes | URLs to crawl (max 3) | |
| base_timeout | No | Timeout per URL (default: 30) | |
| generate_markdown | No | Generate markdown (default: True) | |
| extract_media | No | Extract media (default: False) | |
| wait_for_js | No | Wait for JS (default: False) | |
| output_path | No | Absolute directory path to persist per-URL markdown files + index.json. Existing regular files at this path are rejected; otherwise the directory is created if missing (dot-containing names like /tmp/run.v1 are fine). The list return shape is preserved; each successful item gains an 'output_file' key. Failed items (success=False) are NOT written as .md but still appear in index.json with file=null. | |
| include_content_in_response | No | When True (with output_path), keep full markdown/content in each list item. Defaults to False so the response stays token-efficient. | |
| overwrite | No | Overwrite existing per-URL files inside output_path. Defaults to False (existing files cause an output_path_exists error, returned as a single-element list). |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |