A
licenseB
qualityD
maintenanceA local web scraping MCP server with RAG capabilities that provides intelligent web search, content extraction, and screenshot tools without requiring API keys.
4
12 npm
MIT
freecrawl
by RistrixOP
Provides anti-bot bypass for Akamai-protected websites using patchright, enabling scraping of Akamai-challenged pages.
Provides anti-bot bypass for Cloudflare-protected websites using flaresolverr, enabling scraping of Cloudflare-challenged pages.
Integrates DuckDuckGo search to allow the server to perform web searches as part of the Search feature.
Provides OpenAI-compatible tool definitions for use with LLMs, allowing AI models to invoke scraping, crawling, search, and other actions via function calling. Also supports LLM-based extraction and summarization.
Integrates SearXNG as an alternative search engine for the Search feature, providing flexible search capabilities.
This server does not currently have a deployable release on Glama. Follow the repository's setup instructions to install and configure it for your MCP client, including any required credentials and local dependencies.
🔥 Freecrawl
Open-source web scraping API with anti-bot bypass. Drop-in Firecrawl alternative. AI-first: MCP server, OpenAI tool definitions, REST API — any AI model can use it. No API keys, no credits, no limits.
Quick Start
pip install -r requirements.txt
patchright install chromium
python -m freecrawl serve # REST API on :9100Related MCP server: scraperapi-mcp-server
AI Integration
MCP Server (Claude, Cursor, any MCP client)
{"mcpServers":{"freecrawl":{"command":"python","args":["-m","freecrawl.mcp_server"]}}}OpenAI Tool Definitions
from freecrawl.ai_tools import get_tools
tools = get_tools() # 9 tool definitions, pass to any LLMREST API
curl -X POST http://localhost:9100/v1/scrape \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","formats":["markdown"]}'Features
Feature | Status |
Scrape (URL→Markdown/HTML/text) | ✅ |
Anti-bot bypass (Cloudflare, DataDome) | ✅ |
Auto-escalation engine chain | ✅ |
Crawl (BFS + sitemap) | ✅ |
Search (DuckDuckGo/SearXNG) | ✅ |
Map (fast URL discovery) | ✅ |
Batch scrape (parallel) | ✅ |
LLM extraction (structured JSON) | ✅ |
Change tracking (diff) | ✅ |
PDF parsing | ✅ |
CSS extraction | ✅ |
Screenshots | ✅ |
JS actions (click/scroll/type) | ✅ |
Product/Menu/Branding extractors | ✅ |
Summary (LLM) | ✅ |
Proxy rotation | ✅ |
MCP server | ✅ |
OpenAI tool definitions | ✅ |
Docker deployment | ✅ |
CLI tool | ✅ |
Architecture
Engine Chain (auto-escalation):
httpx (fast) → flaresolverr (Cloudflare) → patchright (DataDome/Akamai)License
MIT
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Related MCP Connectors
Web scraping MCP server — scrape, extract structured data, screenshot any site with anti-bot bypass.
One MCP server for all web data. Scrape, search, crawl, and AI research in one install.
Web scraping, extraction and search for AI agents. Hosted MCP with workspace API keys.
Free web search for AI agents. No API key required. Hosted MCP in active development.
Related MCP Servers
scraperapi-mcp-serverofficial
AlicenseAqualityCmaintenanceA local MCP server that lets AI agents bypass bot detection, geo-restrictions, and JavaScript rendering challenges when scraping the web, backed by ScraperAPI's services285MIT- -licenseNot gradedqualityCmaintenanceSelf-hosted MCP server that provides web scraping and crawling tools, integrating seamlessly with AI frameworks like OpenAI Agents SDK, Cursor, and Claude Code.4-
- AlicenseNot gradedqualityDmaintenanceAn MCP server that gives AI tools a reliable way to fetch content from the web, handling anti-bot protection and JavaScript-rendered pages.1,986 npmMIT