MCP Server Firecrawl

# Firecrawl MCP Server A Model Context Protocol (MCP) server for web scraping, content searching, site crawling, and data extraction using the Firecrawl API. ## Features - **Web Scraping**: Extract content from any webpage with customizable options - Mobile device emulation - Ad and popup blocking - Content filtering - Structured data extraction - Multiple output formats - **Content Search**: Intelligent search capabilities - Multi-language support - Location-based results - Customizable result limits - Structured output formats - **Site Crawling**: Advanced web crawling functionality - Depth control - Path filtering - Rate limiting - Progress tracking - Sitemap integration - **Site Mapping**: Generate site structure maps - Subdomain support - Search filtering - Link analysis - Visual hierarchy - **Data Extraction**: Extract structured data from multiple URLs - Schema validation - Batch processing - Web search enrichment - Custom extraction prompts ## Installation ```bash # Global installation npm install -g @modelcontextprotocol/mcp-server-firecrawl # Local project installation npm install @modelcontextprotocol/mcp-server-firecrawl ``` ## Quick Start 1. Get your Firecrawl API key from the [developer portal](https://firecrawl.dev/dashboard) 2. Set your API key: **Unix/Linux/macOS (bash/zsh):** ```bash export FIRECRAWL_API_KEY=your-api-key ``` **Windows (Command Prompt):** ```cmd set FIRECRAWL_API_KEY=your-api-key ``` **Windows (PowerShell):** ```powershell $env:FIRECRAWL_API_KEY = "your-api-key" ``` **Alternative: Using .env file (recommended for development):** ```bash # Install dotenv npm install dotenv # Create .env file echo "FIRECRAWL_API_KEY=your-api-key" > .env ``` Then in your code: ```javascript import dotenv from 'dotenv'; dotenv.config(); ``` 3. Run the server: ```bash mcp-server-firecrawl ``` ## Integration ### Claude Desktop App Add to your MCP settings: ```json { "firecrawl": { "command": "mcp-server-firecrawl", "env": { "FIRECRAWL_API_KEY": "your-api-key" } } } ``` ### Claude VSCode Extension Add to your MCP configuration: ```json { "mcpServers": { "firecrawl": { "command": "mcp-server-firecrawl", "env": { "FIRECRAWL_API_KEY": "your-api-key" } } } } ``` ## Usage Examples ### Web Scraping ```typescript // Basic scraping { name: "scrape_url", arguments: { url: "https://example.com", formats: ["markdown"], onlyMainContent: true } } // Advanced extraction { name: "scrape_url", arguments: { url: "https://example.com/blog", jsonOptions: { prompt: "Extract article content", schema: { title: "string", content: "string" } }, mobile: true, blockAds: true } } ``` ### Site Crawling ```typescript // Basic crawling { name: "crawl", arguments: { url: "https://example.com", maxDepth: 2, limit: 100 } } // Advanced crawling { name: "crawl", arguments: { url: "https://example.com", maxDepth: 3, includePaths: ["/blog", "/products"], excludePaths: ["/admin"], ignoreQueryParameters: true } } ``` ### Site Mapping ```typescript // Generate site map { name: "map", arguments: { url: "https://example.com", includeSubdomains: true, limit: 1000 } } ``` ### Data Extraction ```typescript // Extract structured data { name: "extract", arguments: { urls: ["https://example.com/product1", "https://example.com/product2"], prompt: "Extract product details", schema: { name: "string", price: "number", description: "string" } } } ``` ## Configuration See [configuration guide](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/docs/configuration.md) for detailed setup options. ## API Documentation See [API documentation](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/docs/api.md) for detailed endpoint specifications. ## Development ```bash # Install dependencies npm install # Build npm run build # Run tests npm test # Start in development mode npm run dev ``` ## Examples Check the [examples](https://github.com/Msparihar/mcp-server-firecrawl/tree/main/examples) directory for more usage examples: - Basic scraping: [scrape.ts](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/examples/scrape.ts) - Crawling and mapping: [crawl-and-map.ts](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/examples/crawl-and-map.ts) ## Error Handling The server implements robust error handling: - Rate limiting with exponential backoff - Automatic retries - Detailed error messages - Debug logging ## Security - API key protection - Request validation - Domain allowlisting - Rate limiting - Safe error messages ## Contributing See [CONTRIBUTING.md](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/CONTRIBUTING.md) for contribution guidelines. ## License MIT License - see [LICENSE](https://github.com/Msparihar/mcp-server-firecrawl/blob/main/LICENSE) for details.