MCP Server Firecrawl

crawl

Crawls a website starting from a base URL

Input Schema

NameRequiredDescriptionDefault
allowBackwardLinksNoAllow crawling links that point to parent directories
allowExternalLinksNoAllow crawling links to external domains
excludePathsNoURL patterns to exclude
ignoreQueryParametersNoIgnore URL query parameters when comparing URLs
ignoreSitemapNoIgnore sitemap.xml during crawling
includePathsNoURL patterns to include
limitNoMaximum pages to crawl
maxDepthNoMaximum crawl depth
scrapeOptionsNoOptions for scraping crawled pages
urlYesBase URL to start crawling from
webhookNoWebhook URL for progress notifications

Input Schema (JSON Schema)

{ "properties": { "allowBackwardLinks": { "description": "Allow crawling links that point to parent directories", "type": "boolean" }, "allowExternalLinks": { "description": "Allow crawling links to external domains", "type": "boolean" }, "excludePaths": { "description": "URL patterns to exclude", "items": { "type": "string" }, "type": "array" }, "ignoreQueryParameters": { "description": "Ignore URL query parameters when comparing URLs", "type": "boolean" }, "ignoreSitemap": { "description": "Ignore sitemap.xml during crawling", "type": "boolean" }, "includePaths": { "description": "URL patterns to include", "items": { "type": "string" }, "type": "array" }, "limit": { "default": 10000, "description": "Maximum pages to crawl", "type": "number" }, "maxDepth": { "default": 2, "description": "Maximum crawl depth", "type": "number" }, "scrapeOptions": { "description": "Options for scraping crawled pages", "type": "object" }, "url": { "description": "Base URL to start crawling from", "type": "string" }, "webhook": { "description": "Webhook URL for progress notifications", "type": "string" } }, "required": [ "url" ], "type": "object" }

You must be authenticated.

Other Tools