Firecrawl Agent MCP Server

# Firecrawl Agent MCP Server A Model Context Protocol (MCP) server that provides AI-powered web data extraction and research capabilities through [Firecrawl's Agent API](https://www.firecrawl.dev/agent). ## Features 🤖 **AI Agent Mode**: Let the agent autonomously search, navigate, and gather data from complex websites 🔍 **Web Search**: Search and scrape multiple results at once 📄 **Single Page Scraping**: Extract content from specific URLs 📊 **Structured Data**: Define JSON schemas for type-safe data extraction 💰 **Cost Control**: Set maximum credit limits per request ⚡ **Async Jobs**: Start long-running tasks and poll for results ## What is Firecrawl Agent? Firecrawl Agent is a magic API that: - **No URLs Required**: Just describe what you need via prompt - **Autonomous Navigation**: Searches and navigates deep into sites to find your data - **Parallel Processing**: Processes multiple sources simultaneously for faster results - **Structured Output**: Returns data in your specified JSON schema format Perfect for: - Research tasks across multiple websites - Extracting structured data (company info, pricing, contacts) - Finding hard-to-reach information - Competitive analysis and market research ## Installation ### 1. Clone or Copy Files ```bash cd firecrawl-agent-mcp ``` ### 2. Install Dependencies ```bash npm install ``` ### 3. Configure API Key Copy the example environment file and add your Firecrawl API key: ```bash cp .env.example .env ``` Edit `.env` and add your API key: ```env FIRECRAWL_API_KEY=fc-YOUR_API_KEY_HERE ``` Get your API key from: https://www.firecrawl.dev/ ### 4. Build the Server ```bash npm run build ``` ## Configuration in Claude Code Add the Firecrawl Agent MCP server to your Claude Code configuration: ### Option 1: Edit `.claude/settings.json` ```json { "mcpServers": { "firecrawl-agent": { "command": "node", "args": ["/absolute/path/to/firecrawl-agent-mcp/dist/server.js"], "env": { "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY_HERE" } } } } ``` ### Option 2: Use `.mcp.json` in Project Root ```json { "mcpServers": { "firecrawl-agent": { "command": "node", "args": ["./firecrawl-agent-mcp/dist/server.js"], "env": { "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY_HERE" } } } } ``` ## Available Tools ### `agent_execute` Execute the AI agent synchronously (waits for completion). **Use when**: You need immediate results for research tasks. **Parameters**: - `prompt` (required): Describe what data you want to extract - `urls` (optional): Specific URLs to search (otherwise searches web) - `schema` (optional): JSON schema for structured output - `maxCredits` (optional): Maximum credits to spend **Example**: ```typescript { "prompt": "Find the founders and founding year of Anthropic", "schema": { "type": "object", "properties": { "founders": { "type": "array", "items": { "type": "string" } }, "founded": { "type": "number" } } } } ``` ### `agent_start` Start an agent job asynchronously (returns job ID immediately). **Use when**: You have long-running research tasks and want to poll for results. **Parameters**: Same as `agent_execute` **Returns**: Job ID to use with `agent_status` ### `agent_status` Check the status of an asynchronous agent job. **Parameters**: - `jobId` (required): Job ID from `agent_start` **Returns**: Current status, progress, and results if completed ### `scrape` Scrape a single URL without AI agent capabilities. **Use when**: You just need to extract content from one specific page. **Parameters**: - `url` (required): URL to scrape - `formats` (optional): Output formats (`markdown`, `html`, `rawHtml`, `links`, `screenshot`) - `onlyMainContent` (optional): Extract only main content (default: true) - `includeTags` (optional): HTML tags to include - `excludeTags` (optional): HTML tags to exclude - `waitFor` (optional): Wait time for JS rendering (ms) - `timeout` (optional): Request timeout (ms) ### `search` Search the web and scrape multiple results. **Use when**: You want to find and extract data from multiple sources at once. **Parameters**: - `query` (required): Search query - `limit` (optional): Maximum number of results (default: 5) - `formats` (optional): Output formats for each result ## Usage Examples ### Example 1: Research Company Information ```typescript // Ask Claude Code: "Use Firecrawl Agent to find information about Anthropic's founding team" // Claude will call: agent_execute({ prompt: "Find the founders of Anthropic and when the company was founded", schema: { type: "object", properties: { founders: { type: "array", items: { type: "string" } }, founded: { type: "number" }, description: { type: "string" } } } }) ``` ### Example 2: Extract Pricing Information ```typescript // Ask Claude Code: "Get pricing information for Claude API" // Claude will call: agent_execute({ prompt: "Extract all pricing tiers and costs for Claude API", urls: ["https://www.anthropic.com/pricing"] }) ``` ### Example 3: Competitive Analysis ```typescript // Ask Claude Code: "Compare the features of the top 5 AI coding assistants" // Claude will call: agent_execute({ prompt: "Find and compare features of top AI coding assistants: GitHub Copilot, Cursor, Claude Code, Tabnine, and Codeium", schema: { type: "object", properties: { tools: { type: "array", items: { type: "object", properties: { name: { type: "string" }, features: { type: "array", items: { type: "string" } }, pricing: { type: "string" } } } } } } }) ``` ### Example 4: Long-Running Research ```typescript // Ask Claude Code: "Start a deep research job on quantum computing breakthroughs in 2024" // Claude will call: const job = await agent_start({ prompt: "Research all major quantum computing breakthroughs and papers published in 2024" }) // Then poll for status: const status = await agent_status({ jobId: job.jobId }) ``` ## Cost Management Firecrawl Agent uses dynamic billing based on task complexity: - Simple extractions: Fewer credits - Complex research: More credits Control costs using: ```typescript { prompt: "Your task", maxCredits: 100 // Limit spending to 100 credits } ``` ## Development ### Watch Mode ```bash npm run dev ``` ### Run Directly ```bash npm start ``` ### SSE Transport Mode For HTTP-based communication: ```bash npm run start:sse ``` ## Troubleshooting ### "FIRECRAWL_API_KEY environment variable is required" Make sure you've: 1. Created a `.env` file with your API key 2. Or configured the env variable in your Claude Code settings ### "HTTP 401: Unauthorized" Your API key is invalid. Get a new one from https://www.firecrawl.dev/ ### "HTTP 429: Too Many Requests" You've hit rate limits. Wait a moment and try again, or upgrade your Firecrawl plan. ### Tools not showing up in Claude Code 1. Make sure you've built the server: `npm run build` 2. Check that the path in your MCP configuration is correct 3. Restart Claude Code after configuration changes ## Learn More - [Firecrawl Agent Documentation](https://docs.firecrawl.dev/features/agent) - [Firecrawl API Reference](https://docs.firecrawl.dev/api-reference/introduction) - [MCP Protocol Specification](https://modelcontextprotocol.io/) - [Introducing Firecrawl Agent Blog Post](https://www.firecrawl.dev/blog/introducing-agent) ## License MIT ## Support For issues with: - **This MCP server**: Open an issue in this repository - **Firecrawl API**: Contact Firecrawl support - **Claude Code**: Visit https://github.com/anthropics/claude-code --- Built with ❤️ using the [Model Context Protocol](https://modelcontextprotocol.io/)