Crawl
crawlDiscover and scrape the pages of a website as one background job, following its links from a starting URL. Returns a crawl id straight away; read progress and results with crawl_status. Each page scraped costs 1 credit, so set limit to what you need.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The site to start from. | |
| limit | No | Maximum pages to scrape. | |
| maxDepth | No | How far from the seed to follow. |