Skip to main content
Glama

Firecrawl site crawl

firecrawl_crawl

Crawl multiple pages from a website URL with configurable path, depth, and page limits, then return the final status and collected page data.

Instructions

Start a multi-page crawl at a website URL, poll it to a terminal state, and return the final status and collected data. Scope can be bounded with include/exclude paths, depth, page limit, subdomain/external-link controls, sitemap handling, delay, and scrape options.

Crawl results can be large; use conservative limits when full-site coverage is unnecessary. Webhooks and interactive scrape actions are unavailable in safe mode. Returns the crawl ID, status, and page data.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "firecrawl_crawl"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYes
delayNo
limitNo
promptNo
sitemapNo
webhookNo
excludePathsNo
includePathsNo
scrapeOptionsNo
maxConcurrencyNo
webhookHeadersNo
allowSubdomainsNo
crawlEntireDomainNo
maxDiscoveryDepthNo
allowExternalLinksNo
ignoreQueryParametersNo
deduplicateSimilarURLsNo

Output Schema

NameRequiredDescriptionDefault
idNoCrawl identifier, for `firecrawl_check_crawl_status`.
urlNoThe URL the crawl started from.
dataNoScraped documents collected by the crawl.
nextNoURL of the next page of results, when the data is paginated.
errorNoError message or error object when the call did not succeed.
totalNoPages the crawl expects to visit.
statusNoCrawl state, such as `scraping`, `completed`, or `failed`.
successNoWhether the API call succeeded.
warningNoNon-fatal warning about the result.
completedNoPages crawled so far.
expiresAtNoWhen the crawl results expire.
agent_hintsNoOptional response guidance from the Firecrawl API.
creditsUsedNoCredits the crawl consumed.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv3.28.2

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, openWorldHint=true, and destructiveHint=false, so the safety profile is partly covered. The description still adds real value by disclosing that the call blocks until a terminal state, that results can be large, and that webhooks and interactive scrape actions are unavailable in safe mode. However, that safe-mode note sits awkwardly against a schema that still accepts 'webhook' and 'scrapeOptions.actions', and the blocking-versus-async model is never fully reconciled with the check_crawl_status sibling, so the picture stays somewhat ambiguous.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences that front-load the core action and then layer scope controls and caveats. Every sentence carries information; the only weakness is that the middle sentence is a dense comma-separated list that is harder to parse than a short bulleted enumeration would be.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so the description needn't detail return values, and its mention of crawl ID, status, and page data is a reasonable pointer. But for a 17-parameter tool with nested objects and an open-world network crawl, the definition leaves the async/polling contract and most parameter meanings unexplained, which is a meaningful gap for an agent trying to size a crawl safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% across 17 parameters, including a deeply nested 'scrapeOptions' object with no field-level documentation anywhere. The description only name-drops categories (include/exclude paths, depth, page limit, subdomain/external-link controls, sitemap, delay, scrape options) without explaining any of their semantics, and leaves 'prompt', 'maxConcurrency', 'webhookHeaders', 'ignoreQueryParameters', and 'deduplicateSimilarURLs' entirely unaddressed. With zero schema coverage, the description had to carry the full explanatory burden and does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb chain plus resource: 'Start a multi-page crawl at a website URL, poll it to a terminal state, and return the final status and collected data.' The word 'multi-page' implicitly separates it from the single-page firecrawl_scrape sibling, but no sibling is named, so the differentiation is left to inference rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The only usage guidance is cautionary ('Crawl results can be large; use conservative limits when full-site coverage is unnecessary'), which is about parameter sizing, not tool selection. Nothing says when to prefer this over firecrawl_scrape, firecrawl_map, or firecrawl_search, and the presence of a firecrawl_check_crawl_status sibling is left unaddressed despite the description claiming the tool polls to a terminal state itself.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.