Firecrawl site crawl
firecrawl_crawlStart a bounded multi-page crawl from a website URL, poll until completion, and return final status with collected page data; control paths, depth, page limits, and scrape options.
Instructions
Start a multi-page crawl at a website URL, poll it to a terminal state, and return the final status and collected data. Scope can be bounded with include/exclude paths, depth, page limit, subdomain/external-link controls, sitemap handling, delay, and scrape options.
Crawl results can be large; use conservative limits when full-site coverage is unnecessary. Webhooks and interactive scrape actions are unavailable in safe mode. Returns the crawl ID, status, and page data.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | ||
| delay | No | ||
| limit | No | ||
| prompt | No | ||
| sitemap | No | ||
| webhook | No | ||
| excludePaths | No | ||
| includePaths | No | ||
| scrapeOptions | No | ||
| maxConcurrency | No | ||
| webhookHeaders | No | ||
| allowSubdomains | No | ||
| crawlEntireDomain | No | ||
| maxDiscoveryDepth | No | ||
| allowExternalLinks | No | ||
| ignoreQueryParameters | No | ||
| deduplicateSimilarURLs | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Crawl identifier, for `firecrawl_check_crawl_status`. | |
| url | No | The URL the crawl started from. | |
| data | No | Scraped documents collected by the crawl. | |
| next | No | URL of the next page of results, when the data is paginated. | |
| error | No | Error message or error object when the call did not succeed. | |
| total | No | Pages the crawl expects to visit. | |
| status | No | Crawl state, such as `scraping`, `completed`, or `failed`. | |
| success | No | Whether the API call succeeded. | |
| warning | No | Non-fatal warning about the result. | |
| completed | No | Pages crawled so far. | |
| expiresAt | No | When the crawl results expire. | |
| creditsUsed | No | Credits the crawl consumed. |