batch_crawl_urls
Crawl up to 10 web pages simultaneously, extracting main text or links. Control concurrency and delay to manage server load and collect data efficiently.
Instructions
Crawl and extract content from multiple web pages
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| urls | Yes | Array of URLs to crawl | |
| extractText | No | Extract main text content | |
| extractLinks | No | Extract all links from pages | |
| maxConcurrent | No | Maximum concurrent requests | |
| delay | No | Delay between batches in milliseconds |