Start Crawl
crawl_startStart crawling a website and immediately return a crawl ID without waiting for completion; use it for asynchronous site crawls, then check status, read results, or cancel early.
Instructions
Start crawling a website and return the crawl id right away, without waiting for it to finish. Check progress with crawl_status, read pages with crawl_results and stop it early with crawl_cancel. Crawled content can overflow your context, so limit defaults to 10 pages. For precise work, prefer map plus scrape_as_markdown on the pages you need.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| geo | No | Geolocation of the desired request, expressed as a country name | |
| url | Yes | Website URL to start crawling from | |
| limit | No | Maximum number of pages to crawl, 10 by default and at most 100 | |
| locale | No | Locale of the desired request | |
| sitemap | No | Whether to use the sitemap alongside links found on pages, only the sitemap, or skip it | |
| maxDepth | No | How many links deep to follow from the start URL | |
| selectPaths | No | Only keep URLs whose path matches one of these patterns, e.g. /blog/* | |
| domainFilter | No | Which links to follow: the same domain, its subdomains too, or any domain | |
| excludePaths | No | Drop URLs whose path matches one of these patterns |