firecrawl_crawl
Crawl websites starting from a URL to discover and scrape linked pages, with configurable limits and path filtering for targeted data extraction.
Instructions
Crawl a website starting from a URL. Discovers and scrapes linked pages. Cost is per page crawled.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Starting URL to crawl | |
| limit | No | Maximum pages to crawl (default 10) | |
| maxDepth | No | Maximum link depth (default 2) | |
| includePaths | No | Glob patterns to include (e.g., ['/blog/*']) | |
| excludePaths | No | Glob patterns to exclude |