firecrawl_crawl
Crawl websites starting from a URL to discover and scrape linked pages, with configurable limits and path filtering for targeted data extraction.
Instructions
Crawl a website starting from a URL. Discovers and scrapes linked pages. Cost is per page crawled.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | Starting URL to crawl | |
| limit | No | Maximum pages to crawl (default 10) | |
| maxDepth | No | Maximum link depth (default 2) | |
| includePaths | No | Glob patterns to include (e.g., ['/blog/*']) | |
| excludePaths | No | Glob patterns to exclude |
Implementation Reference
- src/tools/firecrawl.ts:17-28 (registration)The 'firecrawl_crawl' tool is defined here as part of the 'firecrawlTools' array. It specifies the input schema and the API endpoint.
{ name: "firecrawl_crawl", description: "Crawl a website starting from a URL. Discovers and scrapes linked pages. Cost is per page crawled.", inputSchema: z.object({ url: z.string().describe("Starting URL to crawl"), limit: z.number().optional().describe("Maximum pages to crawl (default 10)"), maxDepth: z.number().optional().describe("Maximum link depth (default 2)"), includePaths: z.array(z.string()).optional().describe("Glob patterns to include (e.g., ['/blog/*'])"), excludePaths: z.array(z.string()).optional().describe("Glob patterns to exclude"), }), endpoint: "/v1/firecrawl/crawl", }, - src/index.ts:15-39 (handler)The 'firecrawl_crawl' tool (and all other tools in 'allTools') is registered in the MCP server here. The handler executes the request using 'gatewayRequest', dynamically hitting the endpoint specified in the tool definition.
for (const tool of allTools) { server.tool( tool.name, tool.description, tool.inputSchema.shape, async (params) => { const method = tool.method || "POST"; const result = await gatewayRequest(method, tool.endpoint, params as Record<string, unknown>); if (result.error) { return { content: [{ type: "text" as const, text: `Error (${result.status}): ${result.error}` }], isError: true, }; } const text = typeof result.data === "string" ? result.data : JSON.stringify(result.data, null, 2); return { content: [{ type: "text" as const, text }], }; }, );