Crawlee MCP
Allows installation directly from GitHub repositories using the npx -y github:patrikmichi/crawlee-mcp command.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Crawlee MCPfetch the contents of https://example.com"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Crawlee MCP
A local Model Context Protocol server for web fetching, crawling, and screenshots with Crawlee. It runs over stdio, so it works with Claude Code, Claude Desktop, Codex, Cursor, and any MCP-compatible client.
Tools
Tool | Description |
| Fetch a single URL and return clean markdown |
| Fetch multiple URLs in parallel |
| Crawl a site, discover links, and return page content |
| Take a PNG screenshot of a page |
| Health check |
Related MCP server: Crawl4AI MCP
Install
Requires Node.js 20 or newer. The first use of JavaScript rendering or screenshots also requires the Chromium browser:
npx playwright install chromiumOnce published to npm, every client can start it with:
npx -y @patrikmichi/crawlee-mcpUntil the npm package is published, install directly from GitHub:
npx -y github:patrikmichi/crawlee-mcpClient configuration
Claude Code
claude mcp add --transport stdio crawlee -- npx -y @patrikmichi/crawlee-mcpClaude Desktop
Add this to your Claude Desktop configuration file:
{
"mcpServers": {
"crawlee": {
"command": "npx",
"args": ["-y", "@patrikmichi/crawlee-mcp"]
}
}
}Codex
Add this to ~/.codex/config.toml:
[mcp_servers.crawlee]
command = "npx"
args = ["-y", "@patrikmichi/crawlee-mcp"]Other MCP clients
Use the same command and arguments: npx -y @patrikmichi/crawlee-mcp.
Configuration
cp env.example .envVariable | Required | Description |
| No | Comma-separated proxy URLs for rotation |
| No | Request timeout in ms (default: |
| No | Set to |
Development
npm install
npm start # Run with tsx (no build needed)
npm run build # Compile TypeScript
npm run typecheck # Type check onlyThis server is intentionally local and stdio-only because JavaScript rendering and screenshots require a local Playwright browser.
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Servers
- Alicense-qualityCmaintenanceEnables web scraping and crawling capabilities for LLM clients, supporting single-page scraping, multi-page website crawling, and web search with multiple engines (Playwright, Cheerio, Puppeteer) and flexible output formats including markdown, HTML, text, and screenshots.186MIT
- AlicenseBqualityAmaintenanceEnables AI assistants to crawl websites, extract dynamic content, navigate links, and save structured Markdown files via the MCP protocol, with support for anti-bot bypass, CSS selectors, and custom JavaScript execution.142MIT
- AlicenseAqualityBmaintenanceEnables web scraping of URLs, including extracting text, links, images, metadata, taking screenshots, and using CSS selectors for structured data extraction.611MIT
- Alicense-qualityCmaintenanceEnables AI agents to fetch and render web pages (including JavaScript-heavy SPAs) with headless Chromium, extract readable content with Mozilla Readability, capture navigation links, download images, and return a clean markdown file path.12ISC
Related MCP Connectors
Read a URL as clean markdown, screenshot a website, url to PDF. Web access for agents, no signup.
Screenshots, PDFs and Markdown from any URL or HTML for AI agents, via the SnapForge API
Web tools for agents: fetch URL as markdown (free MCP) + x402 scrape, links, AI JSON, snapshot.
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/patrikmichi/crawlee-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server