Start a batch scrape
batch_scrapeRun 1-1000 known URLs as one persistent job. Returns { taskId } at once; wait_batch waits for it, get_batch_items pages the results and get_batch_errors lists the failures and robots refusals. Each URL takes scrape's page options (formats, onlyMainContent, includeTags, waitFor, timeout, verify and the rest; their descriptions are on scrape). On hosted Octocrawl a job runs in slices: a crawl reads at most 500 pages and a batch 1,000 URLs, each page costs credits as a scrape does (maxCredits caps the job), and results are kept 7 days after it ends.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"urls": "<urls>"
},
"name": "batch_scrape"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | The declared identity the page is read under: standard (default) or research. A hosted server holds no saved logins, so it has no authed mode. | |
| urls | Yes | ||
| retry | No | softBlocks: extra reads after a 429, 503, other page or failed verify; default 1. | |
| mobile | No | ||
| verify | No | Same as scrape. | |
| formats | No | What to return (default markdown): markdown, links, html, rawHtml, images, tables, json, attributes ({ selectors }), list ({ itemSelector?, fields? }: a record per repeated item, detected without itemSelector), screenshot or screenshot@fullPage (browser only). On an account's credits a browser page costs 5; with fewer left, a screenshot is refused. | |
| headers | No | ||
| timeout | No | ||
| waitFor | No | ||
| blockAds | No | ||
| fastMode | No | ||
| maxCredits | No | The most credits this job may spend; it stops when the next page would pass it. | |
| excludeTags | No | ||
| includeTags | No | ||
| integration | No | Your workflow label; recorded, not sent to sites. | |
| includeLinks | No | ||
| maxFileBytes | No | ||
| idempotencyKey | No | Makes a retry safe: the same key and arguments return the first taskId. Kept 24 hours. | |
| maxConcurrency | No | Pages in flight at once, at most. | |
| onlyMainContent | No | ||
| ignoreInvalidURLs | No | List non-URL entries as invalidURLs instead of refusing the batch. | |
| removeBase64Images | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| taskId | Yes | ||
| replayed | No |