# Firecrawl Agent MCP Server

A Model Context Protocol (MCP) server that provides AI-powered web data extraction and research capabilities through [Firecrawl's Agent API](https://www.firecrawl.dev/agent).

## Features

🤖 **AI Agent Mode**: Let the agent autonomously search, navigate, and gather data from complex websites
🔍 **Web Search**: Search and scrape multiple results at once
📄 **Single Page Scraping**: Extract content from specific URLs
📊 **Structured Data**: Define JSON schemas for type-safe data extraction
💰 **Cost Control**: Set maximum credit limits per request
⚡ **Async Jobs**: Start long-running tasks and poll for results

## What is Firecrawl Agent?

Firecrawl Agent is a magic API that:
- **No URLs Required**: Just describe what you need via prompt
- **Autonomous Navigation**: Searches and navigates deep into sites to find your data
- **Parallel Processing**: Processes multiple sources simultaneously for faster results
- **Structured Output**: Returns data in your specified JSON schema format

Perfect for:
- Research tasks across multiple websites
- Extracting structured data (company info, pricing, contacts)
- Finding hard-to-reach information
- Competitive analysis and market research

## Installation

### 1. Clone or Copy Files

```bash
cd firecrawl-agent-mcp
```

### 2. Install Dependencies

```bash
npm install
```

### 3. Configure API Key

Copy the example environment file and add your Firecrawl API key:

```bash
cp .env.example .env
```

Edit `.env` and add your API key:

```env
FIRECRAWL_API_KEY=fc-YOUR_API_KEY_HERE
```

Get your API key from: https://www.firecrawl.dev/

### 4. Build the Server

```bash
npm run build
```

## Configuration in Claude Code

Add the Firecrawl Agent MCP server to your Claude Code configuration:

### Option 1: Edit `.claude/settings.json`

```json
{
  "mcpServers": {
    "firecrawl-agent": {
      "command": "node",
      "args": ["/absolute/path/to/firecrawl-agent-mcp/dist/server.js"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY_HERE"
      }
    }
  }
}
```

### Option 2: Use `.mcp.json` in Project Root

```json
{
  "mcpServers": {
    "firecrawl-agent": {
      "command": "node",
      "args": ["./firecrawl-agent-mcp/dist/server.js"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY_HERE"
      }
    }
  }
}
```

## Available Tools

### `agent_execute`

Execute the AI agent synchronously (waits for completion).

**Use when**: You need immediate results for research tasks.

**Parameters**:
- `prompt` (required): Describe what data you want to extract
- `urls` (optional): Specific URLs to search (otherwise searches web)
- `schema` (optional): JSON schema for structured output
- `maxCredits` (optional): Maximum credits to spend

**Example**:
```typescript
{
  "prompt": "Find the founders and founding year of Anthropic",
  "schema": {
    "type": "object",
    "properties": {
      "founders": { "type": "array", "items": { "type": "string" } },
      "founded": { "type": "number" }
    }
  }
}
```

### `agent_start`

Start an agent job asynchronously (returns job ID immediately).

**Use when**: You have long-running research tasks and want to poll for results.

**Parameters**: Same as `agent_execute`

**Returns**: Job ID to use with `agent_status`

### `agent_status`

Check the status of an asynchronous agent job.

**Parameters**:
- `jobId` (required): Job ID from `agent_start`

**Returns**: Current status, progress, and results if completed

### `scrape`

Scrape a single URL without AI agent capabilities.

**Use when**: You just need to extract content from one specific page.

**Parameters**:
- `url` (required): URL to scrape
- `formats` (optional): Output formats (`markdown`, `html`, `rawHtml`, `links`, `screenshot`)
- `onlyMainContent` (optional): Extract only main content (default: true)
- `includeTags` (optional): HTML tags to include
- `excludeTags` (optional): HTML tags to exclude
- `waitFor` (optional): Wait time for JS rendering (ms)
- `timeout` (optional): Request timeout (ms)

### `search`

Search the web and scrape multiple results.

**Use when**: You want to find and extract data from multiple sources at once.

**Parameters**:
- `query` (required): Search query
- `limit` (optional): Maximum number of results (default: 5)
- `formats` (optional): Output formats for each result

## Usage Examples

### Example 1: Research Company Information

```typescript
// Ask Claude Code:
"Use Firecrawl Agent to find information about Anthropic's founding team"

// Claude will call:
agent_execute({
  prompt: "Find the founders of Anthropic and when the company was founded",
  schema: {
    type: "object",
    properties: {
      founders: {
        type: "array",
        items: { type: "string" }
      },
      founded: { type: "number" },
      description: { type: "string" }
    }
  }
})
```

### Example 2: Extract Pricing Information

```typescript
// Ask Claude Code:
"Get pricing information for Claude API"

// Claude will call:
agent_execute({
  prompt: "Extract all pricing tiers and costs for Claude API",
  urls: ["https://www.anthropic.com/pricing"]
})
```

### Example 3: Competitive Analysis

```typescript
// Ask Claude Code:
"Compare the features of the top 5 AI coding assistants"

// Claude will call:
agent_execute({
  prompt: "Find and compare features of top AI coding assistants: GitHub Copilot, Cursor, Claude Code, Tabnine, and Codeium",
  schema: {
    type: "object",
    properties: {
      tools: {
        type: "array",
        items: {
          type: "object",
          properties: {
            name: { type: "string" },
            features: { type: "array", items: { type: "string" } },
            pricing: { type: "string" }
          }
        }
      }
    }
  }
})
```

### Example 4: Long-Running Research

```typescript
// Ask Claude Code:
"Start a deep research job on quantum computing breakthroughs in 2024"

// Claude will call:
const job = await agent_start({
  prompt: "Research all major quantum computing breakthroughs and papers published in 2024"
})

// Then poll for status:
const status = await agent_status({ jobId: job.jobId })
```

## Cost Management

Firecrawl Agent uses dynamic billing based on task complexity:
- Simple extractions: Fewer credits
- Complex research: More credits

Control costs using:
```typescript
{
  prompt: "Your task",
  maxCredits: 100  // Limit spending to 100 credits
}
```

## Development

### Watch Mode

```bash
npm run dev
```

### Run Directly

```bash
npm start
```

### SSE Transport Mode

For HTTP-based communication:

```bash
npm run start:sse
```

## Troubleshooting

### "FIRECRAWL_API_KEY environment variable is required"

Make sure you've:
1. Created a `.env` file with your API key
2. Or configured the env variable in your Claude Code settings

### "HTTP 401: Unauthorized"

Your API key is invalid. Get a new one from https://www.firecrawl.dev/

### "HTTP 429: Too Many Requests"

You've hit rate limits. Wait a moment and try again, or upgrade your Firecrawl plan.

### Tools not showing up in Claude Code

1. Make sure you've built the server: `npm run build`
2. Check that the path in your MCP configuration is correct
3. Restart Claude Code after configuration changes

## Learn More

- [Firecrawl Agent Documentation](https://docs.firecrawl.dev/features/agent)
- [Firecrawl API Reference](https://docs.firecrawl.dev/api-reference/introduction)
- [MCP Protocol Specification](https://modelcontextprotocol.io/)
- [Introducing Firecrawl Agent Blog Post](https://www.firecrawl.dev/blog/introducing-agent)

## License

MIT

## Support

For issues with:
- **This MCP server**: Open an issue in this repository
- **Firecrawl API**: Contact Firecrawl support
- **Claude Code**: Visit https://github.com/anthropics/claude-code

---

Built with ❤️ using the [Model Context Protocol](https://modelcontextprotocol.io/)