Skip to main content
Glama
Krieg2065

Firecrawl MCP Server

by Krieg2065

Firecrawl MCP Server

A Model Context Protocol (MCP) server implementation that integrates with Firecrawl for web scraping capabilities.

Big thanks to @vrknetha, @cawstudios for the initial implementation!

You can also play around with our MCP Server on MCP.so's playground. Thanks to MCP.so for hosting and @gstarwd for integrating our server.

Features

  • Scrape, crawl, search, extract, deep research and batch scrape support

  • Web scraping with JS rendering

  • URL discovery and crawling

  • Web search with content extraction

  • Automatic retries with exponential backoff

    • Efficient batch processing with built-in rate limiting

  • Credit usage monitoring for cloud API

  • Comprehensive logging system

  • Support for cloud and self-hosted Firecrawl instances

  • Mobile/Desktop viewport support

  • Smart content filtering with tag inclusion/exclusion

Related MCP server: Firecrawl MCP Server

Installation

Running with npx

env FIRECRAWL_API_KEY=fc-YOUR_API_KEY npx -y firecrawl-mcp

Manual Installation

npm install -g firecrawl-mcp

Running on Cursor

Configuring Cursor 🖥️ Note: Requires Cursor version 0.45.6+ For the most up-to-date configuration instructions, please refer to the official Cursor documentation on configuring MCP servers: Cursor MCP Server Configuration Guide

To configure Firecrawl MCP in Cursor v0.45.6

  1. Open Cursor Settings

  2. Go to Features > MCP Servers

  3. Click "+ Add New MCP Server"

  4. Enter the following:

    • Name: "firecrawl-mcp" (or your preferred name)

    • Type: "command"

    • Command: env FIRECRAWL_API_KEY=your-api-key npx -y firecrawl-mcp

To configure Firecrawl MCP in Cursor v0.48.6

  1. Open Cursor Settings

  2. Go to Features > MCP Servers

  3. Click "+ Add new global MCP server"

  4. Enter the following code:

    {
      "mcpServers": {
        "firecrawl-mcp": {
          "command": "npx",
          "args": ["-y", "firecrawl-mcp"],
          "env": {
            "FIRECRAWL_API_KEY": "YOUR-API-KEY"
          }
        }
      }
    }

If you are using Windows and are running into issues, try cmd /c "set FIRECRAWL_API_KEY=your-api-key && npx -y firecrawl-mcp"

Replace your-api-key with your Firecrawl API key. If you don't have one yet, you can create an account and get it from https://www.firecrawl.dev/app/api-keys

After adding, refresh the MCP server list to see the new tools. The Composer Agent will automatically use Firecrawl MCP when appropriate, but you can explicitly request it by describing your web scraping needs. Access the Composer via Command+L (Mac), select "Agent" next to the submit button, and enter your query.

Running on Windsurf

Add this to your ./codeium/windsurf/model_config.json:

{
  "mcpServers": {
    "mcp-server-firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

Installing via Smithery (Legacy)

To install Firecrawl for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @mendableai/mcp-server-firecrawl --client claude

Configuration

Environment Variables

Required for Cloud API

  • FIRECRAWL_API_KEY: Your Firecrawl API key

    • Required when using cloud API (default)

    • Optional when using self-hosted instance with FIRECRAWL_API_URL

  • FIRECRAWL_API_URL (Optional): Custom API endpoint for self-hosted instances

    • Example: https://firecrawl.your-domain.com

    • If not provided, the cloud API will be used (requires API key)

Optional Configuration

Retry Configuration
  • FIRECRAWL_RETRY_MAX_ATTEMPTS: Maximum number of retry attempts (default: 3)

  • FIRECRAWL_RETRY_INITIAL_DELAY: Initial delay in milliseconds before first retry (default: 1000)

  • FIRECRAWL_RETRY_MAX_DELAY: Maximum delay in milliseconds between retries (default: 10000)

  • FIRECRAWL_RETRY_BACKOFF_FACTOR: Exponential backoff multiplier (default: 2)

Credit Usage Monitoring
  • FIRECRAWL_CREDIT_WARNING_THRESHOLD: Credit usage warning threshold (default: 1000)

  • FIRECRAWL_CREDIT_CRITICAL_THRESHOLD: Credit usage critical threshold (default: 100)

Configuration Examples

For cloud API usage with custom retry and credit monitoring:

# Required for cloud API
export FIRECRAWL_API_KEY=your-api-key

# Optional retry configuration
export FIRECRAWL_RETRY_MAX_ATTEMPTS=5        # Increase max retry attempts
export FIRECRAWL_RETRY_INITIAL_DELAY=2000    # Start with 2s delay
export FIRECRAWL_RETRY_MAX_DELAY=30000       # Maximum 30s delay
export FIRECRAWL_RETRY_BACKOFF_FACTOR=3      # More aggressive backoff

# Optional credit monitoring
export FIRECRAWL_CREDIT_WARNING_THRESHOLD=2000    # Warning at 2000 credits
export FIRECRAWL_CREDIT_CRITICAL_THRESHOLD=500    # Critical at 500 credits

For self-hosted instance:

# Required for self-hosted
export FIRECRAWL_API_URL=https://firecrawl.your-domain.com

# Optional authentication for self-hosted
export FIRECRAWL_API_KEY=your-api-key  # If your instance requires auth

# Custom retry configuration
export FIRECRAWL_RETRY_MAX_ATTEMPTS=10
export FIRECRAWL_RETRY_INITIAL_DELAY=500     # Start with faster retries

Usage with Claude Desktop

Add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "mcp-server-firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "YOUR_API_KEY_HERE",

        "FIRECRAWL_RETRY_MAX_ATTEMPTS": "5",
        "FIRECRAWL_RETRY_INITIAL_DELAY": "2000",
        "FIRECRAWL_RETRY_MAX_DELAY": "30000",
        "FIRECRAWL_RETRY_BACKOFF_FACTOR": "3",

        "FIRECRAWL_CREDIT_WARNING_THRESHOLD": "2000",
        "FIRECRAWL_CREDIT_CRITICAL_THRESHOLD": "500"
      }
    }
  }
}

System Configuration

The server includes several configurable parameters that can be set via environment variables. Here are the default values if not configured:

const CONFIG = {
  retry: {
    maxAttempts: 3, // Number of retry attempts for rate-limited requests
    initialDelay: 1000, // Initial delay before first retry (in milliseconds)
    maxDelay: 10000, // Maximum delay between retries (in milliseconds)
    backoffFactor: 2, // Multiplier for exponential backoff
  },
  credit: {
    warningThreshold: 1000, // Warn when credit usage reaches this level
    criticalThreshold: 100, // Critical alert when credit usage reaches this level
  },
};

These configurations control:

  1. Retry Behavior

    • Automatically retries failed requests due to rate limits

    • Uses exponential backoff to avoid overwhelming the API

    • Example: With default settings, retries will be attempted at:

      • 1st retry: 1 second delay

      • 2nd retry: 2 seconds delay

      • 3rd retry: 4 seconds delay (capped at maxDelay)

  2. Credit Usage Monitoring

    • Tracks API credit consumption for cloud API usage

    • Provides warnings at specified thresholds

    • Helps prevent unexpected service interruption

    • Example: With default settings:

      • Warning at 1000 credits remaining

      • Critical alert at 100 credits remaining

Rate Limiting and Batch Processing

The server utilizes Firecrawl's built-in rate limiting and batch processing capabilities:

  • Automatic rate limit handling with exponential backoff

  • Efficient parallel processing for batch operations

  • Smart request queuing and throttling

  • Automatic retries for transient errors

Available Tools

1. Scrape Tool (firecrawl_scrape)

Scrape content from a single URL with advanced options.

{
  "name": "firecrawl_scrape",
  "arguments": {
    "url": "https://example.com",
    "formats": ["markdown"],
    "onlyMainContent": true,
    "waitFor": 1000,
    "timeout": 30000,
    "mobile": false,
    "includeTags": ["article", "main"],
    "excludeTags": ["nav", "footer"],
    "skipTlsVerification": false
  }
}

2. Batch Scrape Tool (firecrawl_batch_scrape)

Scrape multiple URLs efficiently with built-in rate limiting and parallel processing.

{
  "name": "firecrawl_batch_scrape",
  "arguments": {
    "urls": ["https://example1.com", "https://example2.com"],
    "options": {
      "formats": ["markdown"],
      "onlyMainContent": true
    }
  }
}

Response includes operation ID for status checking:

{
  "content": [
    {
      "type": "text",
      "text": "Batch operation queued with ID: batch_1. Use firecrawl_check_batch_status to check progress."
    }
  ],
  "isError": false
}

3. Check Batch Status (firecrawl_check_batch_status)

Check the status of a batch operation.

{
  "name": "firecrawl_check_batch_status",
  "arguments": {
    "id": "batch_1"
  }
}

4. Search Tool (firecrawl_search)

Search the web and optionally extract content from search results.

{
  "name": "firecrawl_search",
  "arguments": {
    "query": "your search query",
    "limit": 5,
    "lang": "en",
    "country": "us",
    "scrapeOptions": {
      "formats": ["markdown"],
      "onlyMainContent": true
    }
  }
}

5. Crawl Tool (firecrawl_crawl)

Start an asynchronous crawl with advanced options.

{
  "name": "firecrawl_crawl",
  "arguments": {
    "url": "https://example.com",
    "maxDepth": 2,
    "limit": 100,
    "allowExternalLinks": false,
    "deduplicateSimilarURLs": true
  }
}

6. Extract Tool (firecrawl_extract)

Extract structured information from web pages using LLM capabilities. Supports both cloud AI and self-hosted LLM extraction.

{
  "name": "firecrawl_extract",
  "arguments": {
    "urls": ["https://example.com/page1", "https://example.com/page2"],
    "prompt": "Extract product information including name, price, and description",
    "systemPrompt": "You are a helpful assistant that extracts product information",
    "schema": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "price": { "type": "number" },
        "description": { "type": "string" }
      },
      "required": ["name", "price"]
    },
    "allowExternalLinks": false,
    "enableWebSearch": false,
    "includeSubdomains": false
  }
}

Example response:

{
  "content": [
    {
      "type": "text",
      "text": {
        "name": "Example Product",
        "price": 99.99,
        "description": "This is an example product description"
      }
    }
  ],
  "isError": false
}

Extract Tool Options:

  • urls: Array of URLs to extract information from

  • prompt: Custom prompt for the LLM extraction

  • systemPrompt: System prompt to guide the LLM

  • schema: JSON schema for structured data extraction

  • allowExternalLinks: Allow extraction from external links

  • enableWebSearch: Enable web search for additional context

  • includeSubdomains: Include subdomains in extraction

When using a self-hosted instance, the extraction will use your configured LLM. For cloud API, it uses Firecrawl's managed LLM service.

7. Deep Research Tool (firecrawl_deep_research)

Conduct deep web research on a query using intelligent crawling, search, and LLM analysis.

{
  "name": "firecrawl_deep_research",
  "arguments": {
    "query": "how does carbon capture technology work?",
    "maxDepth": 3,
    "timeLimit": 120,
    "maxUrls": 50
  }
}

Arguments:

  • query (string, required): The research question or topic to explore.

  • maxDepth (number, optional): Maximum recursive depth for crawling/search (default: 3).

  • timeLimit (number, optional): Time limit in seconds for the research session (default: 120).

  • maxUrls (number, optional): Maximum number of URLs to analyze (default: 50).

Returns:

  • Final analysis generated by an LLM based on research. (data.finalAnalysis)

  • May also include structured activities and sources used in the research process.

8. Generate LLMs.txt Tool (firecrawl_generate_llmstxt)

Generate a standardized llms.txt (and optionally llms-full.txt) file for a given domain. This file defines how large language models should interact with the site.

{
  "name": "firecrawl_generate_llmstxt",
  "arguments": {
    "url": "https://example.com",
    "maxUrls": 20,
    "showFullText": true
  }
}

Arguments:

  • url (string, required): The base URL of the website to analyze.

  • maxUrls (number, optional): Max number of URLs to include (default: 10).

  • showFullText (boolean, optional): Whether to include llms-full.txt contents in the response.

Returns:

  • Generated llms.txt file contents and optionally the llms-full.txt (data.llmstxt and/or data.llmsfulltxt)

Logging System

The server includes comprehensive logging:

  • Operation status and progress

  • Performance metrics

  • Credit usage monitoring

  • Rate limit tracking

  • Error conditions

Example log messages:

[INFO] Firecrawl MCP Server initialized successfully
[INFO] Starting scrape for URL: https://example.com
[INFO] Batch operation queued with ID: batch_1
[WARNING] Credit usage has reached warning threshold
[ERROR] Rate limit exceeded, retrying in 2s...

Error Handling

The server provides robust error handling:

  • Automatic retries for transient errors

  • Rate limit handling with backoff

  • Detailed error messages

  • Credit usage warnings

  • Network resilience

Example error response:

{
  "content": [
    {
      "type": "text",
      "text": "Error: Rate limit exceeded. Retrying in 2 seconds..."
    }
  ],
  "isError": true
}

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

Contributing

  1. Fork the repository

  2. Create your feature branch

  3. Run tests: npm test

  4. Submit a pull request

License

MIT License - see LICENSE file for details

Available Tools

10 tools
firecrawl_batch_scrapeC

Scrape multiple URLs in batch mode. Returns a job ID that can be used to check status.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlsYesList of URLs to scrape
optionsNo

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that the tool returns a job ID for status checking, which implies an asynchronous operation. However, it doesn't cover critical aspects like rate limits, authentication requirements, error handling, or what happens if URLs fail. For a batch scraping tool with no annotation coverage, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise—two sentences that directly state the tool's function and output. It's front-loaded with the core purpose, and every sentence earns its place by adding value (batch mode and job ID return). No wasted words or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (batch scraping with multiple options and asynchronous behavior), no annotations, and no output schema, the description is incomplete. It doesn't explain the asynchronous nature in detail, error handling, or how to use the job ID with sibling tools like 'firecrawl_check_batch_status.' For a tool with rich input options and no structured safety hints, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 50% (only the 'urls' parameter has a description). The description adds no parameter-specific information beyond what's implied by 'batch mode' and 'job ID.' It doesn't explain the 'options' object or its sub-properties (e.g., 'formats,' 'onlyMainContent'). With low schema coverage, the description fails to compensate adequately, but it doesn't contradict the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Scrape multiple URLs in batch mode.' It specifies the verb (scrape), resource (URLs), and operational mode (batch). However, it doesn't explicitly differentiate from sibling tools like 'firecrawl_scrape' or 'firecrawl_crawl,' which likely handle single URLs or different crawling methods.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides minimal guidance: it mentions batch mode and that it returns a job ID for status checking. However, it doesn't explain when to use this tool versus alternatives like 'firecrawl_scrape' (likely for single URLs) or 'firecrawl_crawl' (possibly for different crawling approaches). No explicit when/when-not scenarios or prerequisites are included.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_check_batch_statusC

Check the status of a batch scraping job.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesBatch job ID to check

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but offers minimal behavioral insight. It states the tool checks status but doesn't describe what status values mean (e.g., pending, completed, failed), whether it's idempotent, or any rate limits. For a status-checking tool, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple status-checking tool and front-loads the key information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what status information is returned (e.g., progress percentage, error messages), how to interpret results, or dependencies on other tools. This leaves the agent with critical gaps in understanding the tool's full context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the single parameter 'id' documented as 'Batch job ID to check'. The description adds no additional meaning beyond this, such as format examples or where to obtain the ID. Given high schema coverage, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('check the status') and resource ('batch scraping job'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'firecrawl_check_crawl_status' (which likely checks individual crawl jobs), but the specificity to 'batch' jobs provides some implicit differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a batch job ID from 'firecrawl_batch_scrape'), exclusions, or comparisons with similar tools like 'firecrawl_check_crawl_status' for non-batch jobs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_check_crawl_statusC

Check the status of a crawl job.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesCrawl job ID to check

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. While 'check' implies a read-only operation, the description doesn't specify what status information is returned, whether there are rate limits, authentication requirements, or what happens with invalid job IDs. This leaves significant gaps for an agent trying to understand the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a simple status-checking tool and front-loads the essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description is insufficiently complete. While the purpose is clear, there's no information about what status information is returned, possible status values, error conditions, or how this differs from the batch status checking sibling tool. The agent would need to guess about the tool's behavior and outputs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% description coverage, with the single parameter 'id' clearly documented as 'Crawl job ID to check'. The description doesn't add any additional semantic context beyond what the schema already provides, but since schema coverage is complete, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('check') and resource ('status of a crawl job'), making the purpose immediately understandable. However, it doesn't distinguish this tool from its sibling 'firecrawl_check_batch_status', which appears to serve a similar status-checking function for batch operations rather than individual crawls.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'firecrawl_check_batch_status' and 'firecrawl_crawl' available, there's no indication whether this is for checking ongoing crawls, completed crawls, or how it differs from batch status checking.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_crawlA

Start an asynchronous crawl of multiple pages from a starting URL. Supports depth control, path filtering, and webhook notifications.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesStarting URL for the crawl
excludePathsNoURL paths to exclude from crawling
includePathsNoOnly crawl these URL paths
maxDepthNoMaximum link depth to crawl
ignoreSitemapNoSkip sitemap.xml discovery
limitNoMaximum number of pages to crawl
allowBackwardLinksNoAllow crawling links that point to parent directories
allowExternalLinksNoAllow crawling links to external domains
webhookNo
deduplicateSimilarURLsNoRemove similar URLs during crawl
ignoreQueryParametersNoIgnore query parameters when comparing URLs
scrapeOptionsNoOptions for scraping each page

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the tool is 'asynchronous' and supports 'webhook notifications,' which are useful behavioral traits. However, it doesn't address critical aspects like rate limits, authentication requirements, error handling, or what happens when the crawl completes (e.g., where results are stored). For a complex tool with 12 parameters, this leaves significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that efficiently conveys the core purpose and key features without unnecessary words. It's front-loaded with the main action and resource, making it easy to understand at a glance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (12 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the basic purpose and some capabilities but lacks details on behavioral traits, return values, or error handling. The high schema coverage helps, but for an asynchronous operation with many options, more context would be beneficial.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is high at 92%, so the baseline is 3 even without parameter details in the description. The description adds minimal value beyond the schema by mentioning 'depth control, path filtering, and webhook notifications,' which loosely correspond to parameters like maxDepth, includePaths/excludePaths, and webhook, but doesn't provide additional semantic context or usage examples.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('start an asynchronous crawl'), the resource ('multiple pages from a starting URL'), and key capabilities ('depth control, path filtering, and webhook notifications'). It distinguishes itself from siblings like firecrawl_scrape or firecrawl_extract by focusing on multi-page crawling rather than single-page operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through 'asynchronous crawl of multiple pages' and mentions capabilities like depth control, which suggests when to use this tool. However, it doesn't explicitly state when to choose this over alternatives like firecrawl_map or firecrawl_batch_scrape, nor does it mention prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_deep_researchC

Conduct deep research on a query using web crawling, search, and AI analysis.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesThe query to research
maxDepthNoMaximum depth of research iterations (1-10)
timeLimitNoTime limit in seconds (30-300)
maxUrlsNoMaximum number of URLs to analyze (1-1000)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'deep research' and methods like 'web crawling, search, and AI analysis', but fails to detail critical behaviors such as rate limits, authentication needs, potential costs, or what 'deep' entails (e.g., iterative analysis, multi-source synthesis). This leaves significant gaps for a tool with complex operations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without unnecessary details. Every word contributes to understanding the tool's function, making it appropriately sized and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity implied by 'deep research' and the lack of annotations and output schema, the description is insufficient. It does not explain what the tool returns (e.g., a report, summarized findings, raw data), how results are formatted, or any error conditions, leaving the agent with incomplete context for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, meaning the input schema already documents all parameters well. The description does not add any additional meaning or context beyond what the schema provides (e.g., it doesn't explain how parameters like 'maxDepth' or 'timeLimit' affect the research process). Thus, it meets the baseline score of 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('conduct deep research') and resources ('on a query'), and mentions the methods used ('web crawling, search, and AI analysis'). However, it does not explicitly differentiate from sibling tools like 'firecrawl_search' or 'firecrawl_crawl', which may have overlapping functionality, preventing a score of 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, such as the sibling tools listed. It lacks explicit instructions on context, prerequisites, or exclusions, leaving the agent to infer usage based on the generic description alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_extractC

Extract structured information from web pages using LLM. Supports both cloud AI and self-hosted LLM extraction.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlsYesList of URLs to extract information from
promptNoPrompt for the LLM extraction
systemPromptNoSystem prompt for LLM extraction
schemaNoJSON schema for structured data extraction
allowExternalLinksNoAllow extraction from external links
enableWebSearchNoEnable web search for additional context
includeSubdomainsNoInclude subdomains in extraction

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool uses LLM for extraction and supports different deployment modes, but lacks critical details: it doesn't specify whether this is a read-only or mutating operation, potential rate limits, authentication needs, error handling, or what the output looks like (since no output schema exists). For a tool with 7 parameters and no annotations, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately concise with two sentences that directly state the tool's core functionality and a key feature. It's front-loaded with the main purpose, and every sentence adds value (the second sentence clarifies deployment options). There's no wasted verbiage or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (7 parameters, no annotations, no output schema), the description is incomplete. It lacks output format details, error conditions, prerequisites (e.g., authentication), and behavioral constraints. For a tool that likely involves network calls and LLM processing, this leaves significant gaps for an agent to use it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 7 parameters thoroughly. The description adds no additional meaning beyond what's in the schema—it doesn't explain parameter interactions, default behaviors, or examples. With high schema coverage, the baseline is 3, as the description doesn't compensate but also doesn't detract.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Extract structured information from web pages using LLM.' It specifies the verb ('extract'), resource ('structured information from web pages'), and method ('using LLM'). However, it doesn't explicitly differentiate from sibling tools like 'firecrawl_scrape' or 'firecrawl_deep_research,' which likely have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It mentions support for 'cloud AI and self-hosted LLM extraction,' but this is a feature detail, not usage context. There are no explicit when/when-not instructions or references to sibling tools, leaving the agent to infer usage scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_generate_llmstxtB

Generate standardized LLMs.txt file for a given URL, which provides context about how LLMs should interact with the website.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to generate LLMs.txt from
maxUrlsNoMaximum number of URLs to process (1-100, default: 10)
showFullTextNoWhether to show the full LLMs-full.txt in the response

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure but provides minimal information. It mentions generating a 'standardized' file but doesn't describe what the generation process entails (e.g., does it crawl the site? analyze content? follow links?), what permissions might be needed, rate limits, or what the output looks like beyond the file name. For a tool with no annotation coverage, this leaves significant behavioral questions unanswered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that clearly states the tool's purpose without unnecessary words. It's appropriately sized for a straightforward tool and front-loads the core functionality ('Generate standardized LLMs.txt file') immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with 3 parameters, 100% schema coverage, but no annotations and no output schema, the description provides adequate basic purpose but lacks important context. It doesn't explain what an LLMs.txt file contains, how it differs from robots.txt, what the generation process involves, or what the agent should expect as a result. The description is minimally complete but leaves significant gaps for effective tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter semantics beyond what's already in the schema descriptions. It mentions 'for a given URL' which aligns with the 'url' parameter but provides no extra context about parameter interactions, defaults, or usage patterns.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Generate standardized LLMs.txt file') and resource ('for a given URL'), with explicit purpose ('provides context about how LLMs should interact with the website'). It distinguishes from sibling tools like 'scrape', 'crawl', or 'extract' by focusing on generating a specific standardized file format rather than general data extraction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'firecrawl_scrape' or 'firecrawl_extract' for similar URL processing tasks. It doesn't mention prerequisites, limitations, or scenarios where this specific LLMs.txt generation is preferred over other data retrieval methods available in the sibling toolset.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_mapB

Discover URLs from a starting point. Can use both sitemap.xml and HTML link discovery.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesStarting URL for URL discovery
searchNoOptional search term to filter URLs
ignoreSitemapNoSkip sitemap.xml discovery and only use HTML links
sitemapOnlyNoOnly use sitemap.xml for discovery, ignore HTML links
includeSubdomainsNoInclude URLs from subdomains in results
limitNoMaximum number of URLs to return

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It mentions discovery methods (sitemap.xml and HTML links) but lacks critical details: it doesn't specify if this is a read-only operation, potential rate limits, authentication needs, output format, or whether it performs recursive crawling. For a discovery tool with 6 parameters, this leaves significant behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly concise with two clear sentences that front-load the core purpose. Every word earns its place: the first sentence states what the tool does, and the second adds important technical context about discovery methods without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a discovery tool with 6 parameters and no annotations or output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., list of URLs, metadata), how results are structured, potential limitations, or error conditions. Given the complexity implied by multiple discovery methods and filtering options, more behavioral context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, providing good parameter documentation. The description adds minimal value beyond the schema by mentioning 'sitemap.xml and HTML link discovery,' which relates to the 'ignoreSitemap' and 'sitemapOnly' parameters. However, it doesn't explain parameter interactions or provide additional context beyond what's already in the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Discover URLs from a starting point' specifies the verb (discover) and resource (URLs). It distinguishes from siblings like 'scrape' or 'extract' by focusing on discovery rather than content extraction. However, it doesn't explicitly differentiate from 'firecrawl_crawl' which might have overlapping functionality.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context by mentioning 'both sitemap.xml and HTML link discovery,' suggesting this tool is for initial URL discovery rather than content processing. However, it doesn't provide explicit guidance on when to use this vs. alternatives like 'firecrawl_crawl' or 'firecrawl_search,' nor does it specify prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

firecrawl_scrapeA

Scrape a single webpage with advanced options for content extraction. Supports various formats including markdown, HTML, and screenshots. Can execute custom actions like clicking or scrolling before scraping.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlYesThe URL to scrape
formatsNoContent formats to extract (default: ['markdown'])
onlyMainContentNoExtract only the main content, filtering out navigation, footers, etc.
includeTagsNoHTML tags to specifically include in extraction
excludeTagsNoHTML tags to exclude from extraction
waitForNoTime in milliseconds to wait for dynamic content to load
timeoutNoMaximum time in milliseconds to wait for the page to load
actionsNoList of actions to perform before scraping
extractNoConfiguration for structured data extraction
mobileNoUse mobile viewport
skipTlsVerificationNoSkip TLS certificate verification
removeBase64ImagesNoRemove base64 encoded images from output
locationNoLocation settings for scraping

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It describes key capabilities (content extraction formats, action execution) and scope (single webpage), but lacks information about rate limits, authentication needs, error handling, or what happens with dynamic content. The mention of 'advanced options' is vague without specifics on limitations or performance characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized with two sentences that efficiently convey core functionality. The first sentence states the primary purpose and key features, while the second adds important behavioral context. There's no wasted language, though it could be slightly more front-loaded with the most critical information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex tool with 13 parameters, nested objects, and no output schema or annotations, the description provides adequate but incomplete context. It covers the 'what' (scraping with options) but lacks information about return values, error conditions, performance expectations, or practical limitations. The absence of output schema means the description should ideally address what the tool returns, which it doesn't.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage, the input schema already documents all 13 parameters thoroughly. The description adds minimal value beyond the schema, mentioning 'advanced options', 'various formats', and 'custom actions' which are already detailed in the schema properties. It doesn't provide additional syntax, examples, or constraints beyond what's in the structured data.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with specific verbs ('scrape', 'extract', 'execute') and resources ('single webpage', 'content extraction', 'custom actions'). It distinguishes from sibling tools by emphasizing 'single webpage' versus batch/crawl operations, and mentions advanced options not implied by the name alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use this tool ('scrape a single webpage with advanced options'), but does not explicitly state when not to use it or name specific alternatives. It implies usage for single-page scraping versus batch operations, but lacks explicit exclusions or comparisons to siblings like firecrawl_extract or firecrawl_crawl.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

TDQS

A3.5/5.0
Disambiguation4/5

Most tools have distinct purposes, such as scrape vs. crawl vs. extract, but there is some potential confusion between firecrawl_batch_scrape and firecrawl_scrape, as both involve scraping with overlapping functionality. The descriptions clarify that batch handles multiple URLs asynchronously, while single scrape is for one page with advanced options, but an agent might still misselect without careful reading.

Naming Consistency5/5

All tool names follow a consistent snake_case pattern with a 'firecrawl_' prefix and descriptive verb_noun combinations, such as firecrawl_scrape, firecrawl_crawl, and firecrawl_extract. This uniformity makes the tool set predictable and easy to navigate for an agent.

Tool Count5/5

With 10 tools, the count is well-scoped for a web crawling and scraping server, covering key operations like scraping, crawling, extracting, searching, and status checks. Each tool appears to earn its place without being excessive or insufficient for the domain.

Completeness4/5

The tool set provides comprehensive coverage for web data extraction, including single and batch scraping, crawling, mapping, searching, and AI-enhanced extraction. A minor gap is the lack of a tool for managing or deleting jobs, but core workflows are well-supported, and agents can likely work around this with the provided status-check tools.

Maintenance

ActivityInactive
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    D
    maintenance
    Integrates Firecrawl web scraping capabilities to extract, crawl, search, and analyze web content with support for batch operations, structured data extraction, and deep research across websites.
    8
    40,139
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Integrates Firecrawl for web scraping, crawling, search, and content extraction capabilities. Supports single/batch scraping, URL discovery, structured data extraction, deep research, and AI-powered web analysis with automatic retries and rate limiting.
    8
    40,139
    MIT
  • A
    license
    A
    quality
    D
    maintenance
    Integrates Firecrawl web scraping capabilities, enabling web content extraction, crawling, site mapping, search, and structured data extraction with automatic rate limiting and retry handling.
    6
    40,139
    2
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/Krieg2065/firecrawl-mcp-server'

If you have feedback or need assistance with the MCP directory API, please join our Discord server