crawl4ai-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@crawl4ai-mcpextract markdown from https://example.com"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
🕷️ Crawl4AI MCP Server
MCP (Model Context Protocol) server for Crawl4AI - Universal web crawling and data extraction for AI agents.
Integrate powerful web scraping capabilities into Claude, ChatGPT, and any MCP-compatible AI assistant.
📑 Table of Contents
Related MCP server: Crawl-MCP
🚀 Quick Start
NPM Installation (Recommended)
# Install globally
npm install -g crawl4ai-mcp-sse-stdio
# Run in different modes
npx crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com
npx crawl4ai-mcp --sse --port 3001 --endpoint https://your-crawl4ai-server.com
npx crawl4ai-mcp --http --port 3000 --endpoint https://your-crawl4ai-server.com
# With optional bearer token
npx crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com --bearer-token your-tokenWith Claude Desktop
Add to your claude_desktop_config.json:
{
"mcpServers": {
"crawl4ai": {
"command": "npx",
"args": [
"crawl4ai-mcp-sse-stdio",
"--stdio",
"--endpoint", "https://your-crawl4ai-server.com",
"--bearer-token", "your-optional-token"
]
}
}
}🐳 Docker Usage
Docker Hub
Official Docker image available on Docker Hub:
# Pull and run the official Docker image
docker pull stgmt/crawl4ai-mcp:latest
docker run -p 3000:3000 stgmt/crawl4ai-mcp:latestDocker Compose
Create a docker-compose.yml:
version: '3.8'
services:
crawl4ai-mcp:
image: stgmt/crawl4ai-mcp:latest
ports:
- "3000:3000"
environment:
- CRAWL4AI_ENDPOINT=https://your-crawl4ai-server.com
- CRAWL4AI_BEARER_TOKEN=your-optional-token
restart: unless-stoppedRun with:
docker-compose up -d🛠️ Available Tools
1. crawl - Full Web Crawling
Extract complete content from any webpage.
{
"name": "crawl",
"arguments": {
"url": "https://example.com",
"wait_for": "css:.content",
"timeout": 30000
}
}2. md - Markdown Extraction
Get clean markdown content from webpages.
{
"name": "md",
"arguments": {
"url": "https://docs.example.com",
"clean": true
}
}3. html - Raw HTML
Retrieve raw HTML content.
{
"name": "html",
"arguments": {
"url": "https://example.com"
}
}4. screenshot - Visual Capture
Take screenshots of webpages.
{
"name": "screenshot",
"arguments": {
"url": "https://example.com",
"full_page": true
}
}5. pdf - PDF Generation
Convert webpages to PDF.
{
"name": "pdf",
"arguments": {
"url": "https://example.com",
"format": "A4"
}
}6. execute_js - JavaScript Execution
Execute JavaScript on webpages.
{
"name": "execute_js",
"arguments": {
"url": "https://example.com",
"script": "document.title"
}
}⚙️ Configuration
Environment Variables
# REQUIRED: Crawl4AI endpoint URL
export CRAWL4AI_ENDPOINT="https://your-crawl4ai-server.com"
# OPTIONAL: Bearer authentication token
export CRAWL4AI_BEARER_TOKEN="your-api-token"Command Line Options
crawl4ai-mcp --help
Options:
--stdio Run in STDIO mode for MCP clients
--sse Run in SSE mode for web interfaces
--http Run in HTTP mode
--endpoint ENDPOINT Crawl4AI API endpoint URL (REQUIRED)
--bearer-token TOKEN Bearer authentication token (OPTIONAL)
--port PORT HTTP server port (default: 3000)
--sse-port PORT SSE server port (default: 9001)
--version, -v Show versionBasic Commands
# HTTP mode (recommended for testing)
crawl4ai-mcp --http --port 3000 --endpoint https://your-crawl4ai-server.com
# SSE mode (Server-Sent Events)
crawl4ai-mcp --sse --port 3001 --endpoint https://your-crawl4ai-server.com
# STDIO mode (for MCP clients)
crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com
# With optional bearer token
crawl4ai-mcp --http --port 3000 --endpoint https://your-crawl4ai-server.com --bearer-token your-token🤝 Contributing
We welcome contributions! See CONTRIBUTING.md for guidelines.
📄 License
MIT License - see LICENSE file for details.
🔗 Links
NPM Package: https://www.npmjs.com/package/crawl4ai-mcp-sse-stdio
GitHub Repository: https://github.com/stgmt/crawl4ai-mcp
Docker Hub: https://hub.docker.com/r/stgmt/crawl4ai-mcp
Made with ❤️ for the AI community
This server cannot be deployed
Maintenance
Related MCP Connectors
Turn any public website into an MCP server for agents to search, read and navigate.
MCP server for Firecrawl — web search, scraping, and biomedical/arXiv paper search.
Scrape, crawl and search the web for AI agents via MCP.
Official MCP server for Agentwork — delegate tasks to AI agents with human-in-the-loop
Related MCP Servers
- FlicenseBqualityDmaintenanceAn MCP Server for Web scraping and Crawling, built using Crawl4AI224-
- AlicenseAqualityBmaintenanceUnofficial MCP server wrapping crawl4ai that enables extraction and analysis of content from web pages, PDFs, Office documents, YouTube videos, and more, with AI-powered summarization and Google search integration to reduce token usage while preserving key information.1948MIT
- AlicenseNot gradedqualityDmaintenanceAn MCP server that integrates Crawl4AI with Supabase to enable AI agents to crawl websites, store content in a vector database, and perform RAG queries.8MIT
- AlicenseNot gradedqualityFmaintenanceA lightweight MCP server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents, enabling single-page scraping and multi-page crawling with adaptive stopping.114MIT