freecrawl
Provides anti-bot bypass for Akamai-protected websites using patchright, enabling scraping of Akamai-challenged pages.
Provides anti-bot bypass for Cloudflare-protected websites using flaresolverr, enabling scraping of Cloudflare-challenged pages.
Integrates DuckDuckGo search to allow the server to perform web searches as part of the Search feature.
Provides OpenAI-compatible tool definitions for use with LLMs, allowing AI models to invoke scraping, crawling, search, and other actions via function calling. Also supports LLM-based extraction and summarization.
Integrates SearXNG as an alternative search engine for the Search feature, providing flexible search capabilities.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@freecrawlscrape https://example.com to markdown"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
🔥 Freecrawl
Open-source web scraping API with anti-bot bypass. Drop-in Firecrawl alternative. AI-first: MCP server, OpenAI tool definitions, REST API — any AI model can use it. No API keys, no credits, no limits.
Quick Start
pip install -r requirements.txt
patchright install chromium
python -m freecrawl serve # REST API on :9100Related MCP server: ISIS MCP
AI Integration
MCP Server (Claude, Cursor, any MCP client)
{"mcpServers":{"freecrawl":{"command":"python","args":["-m","freecrawl.mcp_server"]}}}OpenAI Tool Definitions
from freecrawl.ai_tools import get_tools
tools = get_tools() # 9 tool definitions, pass to any LLMREST API
curl -X POST http://localhost:9100/v1/scrape \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","formats":["markdown"]}'Features
Feature | Status |
Scrape (URL→Markdown/HTML/text) | ✅ |
Anti-bot bypass (Cloudflare, DataDome) | ✅ |
Auto-escalation engine chain | ✅ |
Crawl (BFS + sitemap) | ✅ |
Search (DuckDuckGo/SearXNG) | ✅ |
Map (fast URL discovery) | ✅ |
Batch scrape (parallel) | ✅ |
LLM extraction (structured JSON) | ✅ |
Change tracking (diff) | ✅ |
PDF parsing | ✅ |
CSS extraction | ✅ |
Screenshots | ✅ |
JS actions (click/scroll/type) | ✅ |
Product/Menu/Branding extractors | ✅ |
Summary (LLM) | ✅ |
Proxy rotation | ✅ |
MCP server | ✅ |
OpenAI tool definitions | ✅ |
Docker deployment | ✅ |
CLI tool | ✅ |
Architecture
Engine Chain (auto-escalation):
httpx (fast) → flaresolverr (Cloudflare) → patchright (DataDome/Akamai)License
MIT
This server cannot be installed
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Latest Blog Posts
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/RistrixOP/freecrawl'
If you have feedback or need assistance with the MCP directory API, please join our Discord server