crawl
Start an asynchronous crawl from a seed URL, returning every page as Markdown. Obtain a job ID immediately and track progress with crawl_status.
Instructions
Start an asynchronous BFS crawl from a seed URL; each page comes back as Markdown. Returns a job id immediately — poll with crawl_status. $0.0003 per page, unfetched budget refunded. Prefer map + batch when you can filter the URL list first.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Seed URL. | |
| depth | No | Max link depth from the seed, default 3, cap 10. | |
| limit | No | Max pages, default 50, cap 500. | |
| render | No | Render every page with the browser (much slower). | |
| country | No | ISO country code for the proxy exit. | |
| exclude | No | URL substrings to skip. | |
| include | No | URL substrings to include. | |
| contentMode | No | Content scope per page. |