MCP Server Firecrawl
by Msparihar
crawl
Crawls a website starting from a base URL
Input Schema
Name | Required | Description | Default |
---|---|---|---|
allowBackwardLinks | No | Allow crawling links that point to parent directories | |
allowExternalLinks | No | Allow crawling links to external domains | |
excludePaths | No | URL patterns to exclude | |
ignoreQueryParameters | No | Ignore URL query parameters when comparing URLs | |
ignoreSitemap | No | Ignore sitemap.xml during crawling | |
includePaths | No | URL patterns to include | |
limit | No | Maximum pages to crawl | |
maxDepth | No | Maximum crawl depth | |
scrapeOptions | No | Options for scraping crawled pages | |
url | Yes | Base URL to start crawling from | |
webhook | No | Webhook URL for progress notifications |