batch_scrape
Scrape 2 to 50 URLs in parallel to collect product pages, news articles, or competitor data. Use sync mode for immediate results or async with webhook for large batches.
Instructions
Use this when you need to scrape 2–50 URLs in parallel — e.g. batch-collecting product pages, news articles, or competitor pages. Use mode:"async" with a webhook for large batches; mode:"sync" for up to ~25 URLs when you need results immediately. Example: batch_scrape({urls: ["https://a.com","https://b.com"], formats: ["json"], maxConcurrency: 5})
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mode | No | Processing mode: sync (wait) or async (background) | sync |
| urls | Yes | Array of URLs or URL objects to scrape | |
| formats | No | Output formats for scraped content | |
| webhook | No | Webhook configuration for async job notifications | |
| pageSize | No | Number of results per page | |
| jobOptions | No | Job management options for async processing | |
| includeFailed | No | Include failed URLs in results | |
| maxConcurrency | No | Maximum concurrent scraping requests | |
| includeMetadata | No | Include page metadata in results | |
| extractionSchema | No | Schema for structured data extraction from each URL | |
| delayBetweenRequests | No | Delay in milliseconds between requests |