A
licenseB
qualityD
maintenanceA local web scraping MCP server with RAG capabilities that provides intelligent web search, content extraction, and screenshot tools without requiring API keys.
4
7 npm
MIT
freecrawl
by RistrixOP
Provides anti-bot bypass for Akamai-protected websites using patchright, enabling scraping of Akamai-challenged pages.
Provides anti-bot bypass for Cloudflare-protected websites using flaresolverr, enabling scraping of Cloudflare-challenged pages.
Integrates DuckDuckGo search to allow the server to perform web searches as part of the Search feature.
Provides OpenAI-compatible tool definitions for use with LLMs, allowing AI models to invoke scraping, crawling, search, and other actions via function calling. Also supports LLM-based extraction and summarization.
Integrates SearXNG as an alternative search engine for the Search feature, providing flexible search capabilities.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@freecrawlscrape https://example.com to markdown"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
🔥 Freecrawl
Open-source web scraping API with anti-bot bypass. Drop-in Firecrawl alternative. AI-first: MCP server, OpenAI tool definitions, REST API — any AI model can use it. No API keys, no credits, no limits.
Quick Start
pip install -r requirements.txt
patchright install chromium
python -m freecrawl serve # REST API on :9100Related MCP server: scraperapi-mcp-server
AI Integration
MCP Server (Claude, Cursor, any MCP client)
{"mcpServers":{"freecrawl":{"command":"python","args":["-m","freecrawl.mcp_server"]}}}OpenAI Tool Definitions
from freecrawl.ai_tools import get_tools
tools = get_tools() # 9 tool definitions, pass to any LLMREST API
curl -X POST http://localhost:9100/v1/scrape \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","formats":["markdown"]}'Features
Feature | Status |
Scrape (URL→Markdown/HTML/text) | ✅ |
Anti-bot bypass (Cloudflare, DataDome) | ✅ |
Auto-escalation engine chain | ✅ |
Crawl (BFS + sitemap) | ✅ |
Search (DuckDuckGo/SearXNG) | ✅ |
Map (fast URL discovery) | ✅ |
Batch scrape (parallel) | ✅ |
LLM extraction (structured JSON) | ✅ |
Change tracking (diff) | ✅ |
PDF parsing | ✅ |
CSS extraction | ✅ |
Screenshots | ✅ |
JS actions (click/scroll/type) | ✅ |
Product/Menu/Branding extractors | ✅ |
Summary (LLM) | ✅ |
Proxy rotation | ✅ |
MCP server | ✅ |
OpenAI tool definitions | ✅ |
Docker deployment | ✅ |
CLI tool | ✅ |
Architecture
Engine Chain (auto-escalation):
httpx (fast) → flaresolverr (Cloudflare) → patchright (DataDome/Akamai)License
MIT
This server cannot be deployed
Maintenance
ActivityStale
ResponsivenessNo issues
Related MCP Connectors
Zenrows MCP server — Fetch, Extract, Batch, and Browser Sessions for AI coding assistants
One MCP for the Web. Easily search, crawl, navigate, and extract websites without getting blocked.…
All HasData scraping tools in one MCP server: Google, TikTok, Instagram, maps, e-commerce and more.
Scrape, crawl and search the web for AI agents via MCP.
Related MCP Servers
scraperapi-mcp-serverofficial
AlicenseAqualityBmaintenanceA local MCP server that lets AI agents bypass bot detection, geo-restrictions, and JavaScript rendering challenges when scraping the web, backed by ScraperAPI's services285MIT- -licenseNot gradedqualityCmaintenanceSelf-hosted MCP server that provides web scraping and crawling tools, integrating seamlessly with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code.4-
- AlicenseNot gradedqualityDmaintenanceAn MCP server that gives AI tools a reliable way to fetch content from the web, handling anti-bot protection and JavaScript-rendered pages.730 npmMIT