crawl4ai-mcp
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
🕷️ Crawl4AI MCP Server
MCP (Model Context Protocol) server for Crawl4AI - Universal web crawling and data extraction for AI agents.
Integrate powerful web scraping capabilities into Claude, ChatGPT, and any MCP-compatible AI assistant.
📑 Table of Contents
Related MCP server: Crawl-MCP
🚀 Quick Start
NPM Installation (Recommended)
# Install globally
npm install -g crawl4ai-mcp-sse-stdio
# Run in different modes
npx crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com
npx crawl4ai-mcp --sse --port 3001 --endpoint https://your-crawl4ai-server.com
npx crawl4ai-mcp --http --port 3000 --endpoint https://your-crawl4ai-server.com
# With optional bearer token
npx crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com --bearer-token your-tokenWith Claude Desktop
Add to your claude_desktop_config.json:
{
"mcpServers": {
"crawl4ai": {
"command": "npx",
"args": [
"crawl4ai-mcp-sse-stdio",
"--stdio",
"--endpoint", "https://your-crawl4ai-server.com",
"--bearer-token", "your-optional-token"
]
}
}
}🐳 Docker Usage
Docker Hub
Official Docker image available on Docker Hub:
# Pull and run the official Docker image
docker pull stgmt/crawl4ai-mcp:latest
docker run -p 3000:3000 stgmt/crawl4ai-mcp:latestDocker Compose
Create a docker-compose.yml:
version: '3.8'
services:
crawl4ai-mcp:
image: stgmt/crawl4ai-mcp:latest
ports:
- "3000:3000"
environment:
- CRAWL4AI_ENDPOINT=https://your-crawl4ai-server.com
- CRAWL4AI_BEARER_TOKEN=your-optional-token
restart: unless-stoppedRun with:
docker-compose up -d🛠️ Available Tools
1. crawl - Full Web Crawling
Extract complete content from any webpage.
{
"name": "crawl",
"arguments": {
"url": "https://example.com",
"wait_for": "css:.content",
"timeout": 30000
}
}2. md - Markdown Extraction
Get clean markdown content from webpages.
{
"name": "md",
"arguments": {
"url": "https://docs.example.com",
"clean": true
}
}3. html - Raw HTML
Retrieve raw HTML content.
{
"name": "html",
"arguments": {
"url": "https://example.com"
}
}4. screenshot - Visual Capture
Take screenshots of webpages.
{
"name": "screenshot",
"arguments": {
"url": "https://example.com",
"full_page": true
}
}5. pdf - PDF Generation
Convert webpages to PDF.
{
"name": "pdf",
"arguments": {
"url": "https://example.com",
"format": "A4"
}
}6. execute_js - JavaScript Execution
Execute JavaScript on webpages.
{
"name": "execute_js",
"arguments": {
"url": "https://example.com",
"script": "document.title"
}
}⚙️ Configuration
Environment Variables
# REQUIRED: Crawl4AI endpoint URL
export CRAWL4AI_ENDPOINT="https://your-crawl4ai-server.com"
# OPTIONAL: Bearer authentication token
export CRAWL4AI_BEARER_TOKEN="your-api-token"Command Line Options
crawl4ai-mcp --help
Options:
--stdio Run in STDIO mode for MCP clients
--sse Run in SSE mode for web interfaces
--http Run in HTTP mode
--endpoint ENDPOINT Crawl4AI API endpoint URL (REQUIRED)
--bearer-token TOKEN Bearer authentication token (OPTIONAL)
--port PORT HTTP server port (default: 3000)
--sse-port PORT SSE server port (default: 9001)
--version, -v Show versionBasic Commands
# HTTP mode (recommended for testing)
crawl4ai-mcp --http --port 3000 --endpoint https://your-crawl4ai-server.com
# SSE mode (Server-Sent Events)
crawl4ai-mcp --sse --port 3001 --endpoint https://your-crawl4ai-server.com
# STDIO mode (for MCP clients)
crawl4ai-mcp --stdio --endpoint https://your-crawl4ai-server.com
# With optional bearer token
crawl4ai-mcp --http --port 3000 --endpoint https://your-crawl4ai-server.com --bearer-token your-token🤝 Contributing
We welcome contributions! See CONTRIBUTING.md for guidelines.
📄 License
MIT License - see LICENSE file for details.
🔗 Links
NPM Package: https://www.npmjs.com/package/crawl4ai-mcp-sse-stdio
GitHub Repository: https://github.com/stgmt/crawl4ai-mcp
Docker Hub: https://hub.docker.com/r/stgmt/crawl4ai-mcp
Made with ❤️ for the AI community
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
This server cannot be deployed
Maintenance
Related MCP Connectors
One MCP server for all web data. Scrape, search, crawl, and AI research in one install.
Turn any public website into an MCP server for agents to search, read and navigate.
Hosted Web Crawler MCP server for Claude, ChatGPT and Cursor, with each action switched per account.
Web scraping, browser automation, screenshots and AI data extraction for AI agents via MCP.
Related MCP Servers
- FlicenseBqualityDmaintenanceAn MCP Server for Web scraping and Crawling, built using Crawl4AI224-
- AlicenseBqualityBmaintenanceUnofficial MCP server wrapping crawl4ai that enables extraction and analysis of content from web pages, PDFs, Office documents, YouTube videos, and more, with AI-powered summarization and Google search integration to reduce token usage while preserving key information.2148MIT
- AlicenseNot gradedqualityDmaintenanceAn MCP server that integrates Crawl4AI with Supabase to enable AI agents to crawl websites, store content in a vector database, and perform RAG queries.8MIT
- AlicenseNot gradedqualityDmaintenanceA lightweight MCP server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents, enabling single-page scraping and multi-page crawling with adaptive stopping.118MIT