Skip to main content
Glama
Firecrawl MCP Server

Firecrawl MCP Server

Firecrawl site crawl

firecrawl_crawl

Start a bounded multi-page crawl from a website URL, poll until completion, and return final status with collected page data; control paths, depth, page limits, and scrape options.

Instructions

Start a multi-page crawl at a website URL, poll it to a terminal state, and return the final status and collected data. Scope can be bounded with include/exclude paths, depth, page limit, subdomain/external-link controls, sitemap handling, delay, and scrape options.

Crawl results can be large; use conservative limits when full-site coverage is unnecessary. Webhooks and interactive scrape actions are unavailable in safe mode. Returns the crawl ID, status, and page data.

Input Schema

NameRequiredDescriptionDefault
urlYes
delayNo
limitNo
promptNo
sitemapNo
webhookNo
excludePathsNo
includePathsNo
scrapeOptionsNo
maxConcurrencyNo
webhookHeadersNo
allowSubdomainsNo
crawlEntireDomainNo
maxDiscoveryDepthNo
allowExternalLinksNo
ignoreQueryParametersNo
deduplicateSimilarURLsNo

Output Schema

NameRequiredDescriptionDefault
idNoCrawl identifier, for `firecrawl_check_crawl_status`.
urlNoThe URL the crawl started from.
dataNoScraped documents collected by the crawl.
nextNoURL of the next page of results, when the data is paginated.
errorNoError message or error object when the call did not succeed.
totalNoPages the crawl expects to visit.
statusNoCrawl state, such as `scraping`, `completed`, or `failed`.
successNoWhether the API call succeeded.
warningNoNon-fatal warning about the result.
completedNoPages crawled so far.
expiresAtNoWhen the crawl results expire.
creditsUsedNoCredits the crawl consumed.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changedv3.36.1
    • changedInput schema / properties / scrapeOptions / properties / jsonOptions / properties / schema / additionalProperties
      Previous value: -{}New value: +true
  2. Changed5 schema fields changedv1.0.2
    • changedInput schema / $schema
      Previous value: -"https://json-schema.org/draft/2020-12/schema"New value: +"http://json-schema.org/draft-07/schema#"
    • removedInput schema / properties / scrapeOptions / properties / actions / items / additionalProperties
      Removed value: -false
    • changedInput schema / properties / scrapeOptions / properties / formats / items / enum
      Previous value: -[
      -  "markdown",
      -  "html",
      -  "rawHtml",
      -  "screenshot",
      -  "links",
      -  "summary",
      -  "changeTracking",
      -  "branding",
      -  "json",
      -  "query",
      -  "audio"
      -]New value: +[
      +  "markdown",
      +  "html",
      +  "rawHtml",
      +  "screenshot",
      +  "links",
      +  "images",
      +  "summary",
      +  "changeTracking",
      +  "branding",
      +  "json",
      +  "query",
      +  "audio"
      +]
    • changedInput schema / properties / scrapeOptions / properties / queryOptions / required
      Previous value: -[
      -  "prompt",
      -  "mode"
      -]New value: +[
      +  "prompt"
      +]
    • changedOutput schema / (root)
      Previous value: -nullNew value: +{
      +  "$schema": "http://json-schema.org/draft-07/schema#",
      +  "additionalProperties": false,
      +  "description": "Crawl job state and the documents collected so far.",
      +  "properties": {
      +    "completed": {
      +      "anyOf": [
      +        {
      +          "type": "number"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "Pages crawled so far."
      +    },
      +    "creditsUsed": {
      +      "anyOf": [
      +        {
      +          "type": "number"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "Credits the crawl consumed."
      +    },
      +    "data": {
      +      "description": "Scraped documents collected by the crawl."
      +    },
      +    "error": {
      +      "description": "Error message or error object when the call did not succeed."
      +    },
      +    "expiresAt": {
      +      "anyOf": [
      +        {
      +          "type": "string"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "When the crawl results expire."
      +    },
      +    "id": {
      +      "anyOf": [
      +        {
      +          "type": "string"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "Crawl identifier, for `firecrawl_check_crawl_status`."
      +    },
      +    "next": {
      +      "anyOf": [
      +        {
      +          "type": "string"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "URL of the next page of results, when the data is paginated."
      +    },
      +    "status": {
      +      "anyOf": [
      +        {
      +          "type": "string"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "Crawl state, such as `scraping`, `completed`, or `failed`."
      +    },
      +    "success": {
      +      "anyOf": [
      +        {
      +          "type": "boolean"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "Whether the API call succeeded."
      +    },
      +    "total": {
      +      "anyOf": [
      +        {
      +          "type": "number"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "Pages the crawl expects to visit."
      +    },
      +    "url": {
      +      "anyOf": [
      +        {
      +          "type": "string"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "The URL the crawl started from."
      +    },
      +    "warning": {
      +      "anyOf": [
      +        {
      +          "type": "string"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ],
      +      "description": "Non-fatal warning about the result."
      +    }
      +  },
      +  "type": "object"
      +}
  3. First observedv1.0.1

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations only cover the safety profile (readOnlyHint=false, openWorldHint=true, non-idempotent), while the description adds the important traits the agent actually needs: the call 'polls it to a terminal state' (that is, it blocks until completion) and results 'can be large.' It also calls out safe-mode restrictions that silently disable certain parameters. Missing only auth/cost and whether the crawl is cancellable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight paragraphs with the core action and scope controls front-loaded and the cost warning before the return-value line. The closing sentence partially duplicates the opening ('return the final status and collected data' vs 'Returns the crawl ID, status, and page data'), which is the only real waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given a very large nested schema and an existing output schema, the description still covers the agent-critical facts: that this is a long, blocking, multi-page operation whose output scales with scope. It leaves adjacency questions (when to use this versus scrape or check status) unanswered, which is the main gap for a tool this complex.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 17 parameters, nested scrapeOptions, and 0% schema description coverage, the description's group-by-group walkthrough (include/exclude paths, depth, limit, subdomain/external-link controls, sitemap, delay, scrape options) is the only semantic signal available and does help mapping intent to knobs. But it gives no meaning for values or defaults and never mentions prompt, webhookHeaders, maxConcurrency, ignoreQueryParameters, or deduplicateSimilarURLs, so roughly half the surface stays opaque.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Start a multi-page crawl at a website URL.' 'Multi-page' implicitly separates it from firecrawl_scrape, but no sibling (firecrawl_scrape, firecrawl_map, firecrawl_check_crawl_status) is named explicitly, so a reader must infer the boundary from the word 'multi-page'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives one actionable usage constraint, 'use conservative limits when full-site coverage is unnecessary,' and notes that webhooks and interactive actions are unavailable in safe mode. It never says when to prefer firecrawl_scrape for a single page, firecrawl_map for URL discovery, or firecrawl_check_crawl_status, so alternative routing remains implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.