chameleon
The chameleon server is an AI-agent-friendly web scraping and crawling toolkit with advanced anti-bot countermeasures. It provides:
Scrape single URL (
scrape_url): Fetch and convert to Markdown/JSON, with rendering mode, proxy region, wait conditions, and structured data extraction via prompt or schema.Deep crawl site (
crawl_site): Asynchronously crawl entire websites using BFS/DFS/Adaptive strategies, configurable depth/limits; monitor withget_job_status.Map website URLs (
map_site): Discover all URLs on a site, optionally using sitemap only.Extract structured data (
extract_data): Use JSON Schema to extract specific fields (prices, titles, etc.) with configurable extraction strategy.Search web (
search_web): Perform internet searches and get results in Markdown.Batch scrape (
batch_scrape): Concurrently fetch multiple URLs with configurable output format.Screenshot (
get_screenshot): Capture full-page or viewport screenshots as base64 PNG.Diagnose anti-bot (
diagnose_site): Analyze a site's anti-scraping measures and get recommended strategies.Check proxy (
check_proxy): Verify proxy IP connectivity and exit information.Robots.txt (
get_robots_txt): Fetch a site's robots.txt rules.Network log (
get_network_log): Capture XHR/fetch requests for API discovery.Advanced anti-scraping: Automatically escalates through 6 levels of evasion techniques including UA/header spoofing, proxy rotation, TLS fingerprint simulation, and full browser rendering with behavior simulation.
All operations are designed for seamless AI agent integration.
Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@chameleonScrape https://example.com/products and extract product names and prices as JSON"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Chameleon 🦎
面向真实世界反爬、动态渲染、异常容错的 AI Agent 爬虫工具。提供 MCP Server + REST API + CLI 三套 Agent 友好接口,核心理念:分层对抗、自适应降级、Agent 友好输出。
核心能力
6 级反爬自动升级:裸请求 → UA/Header 伪装 → 代理轮换 → TLS 指纹模拟 → 浏览器渲染 → 行为模拟/验证码,逐级升级,全程可观测
三引擎架构:HTTP 引擎(httpx/curl_cffi)、浏览器引擎(Playwright + stealth + 行为模拟)、API 逆向引擎
处理管线:HTML 清洗 → Markdown 转换(token 预算裁剪)→ 结构化提取(CSS/XPath/LLM/Hybrid/表格)→ 动态 Pydantic 校验
深度爬取:BFS/DFS/Adaptive 策略、每域名限速、ETag 增量采集、断点续爬、robots.txt 合规
Agent 原生:MCP 12 工具自动发现、统一结构化输出(status/content/metadata)、SSRF 防护 + 审计日志
Related MCP server: Selenium MCP Server
接口形态
形态 | 说明 |
MCP Server | stdio 传输,12 个工具,SSRF 防护 + 输出截断 |
REST API | FastAPI,与 MCP 工具一一对应,API Key 认证 + WebSocket 进度 |
CLI |
|
Python SDK |
|
快速开始
uv sync
uv run chameleon scrape https://example.com -o markdown # CLI
uv run python -m chameleon.interfaces.mcp_server # MCP (stdio)
uv run uvicorn chameleon.interfaces.rest_api:app --port 8000 # REST APIMCP 配置(Claude Code / opencode)
{
"mcpServers": {
"chameleon": {
"command": "uv",
"args": ["run", "python", "-m", "chameleon.interfaces.mcp_server"]
}
}
}Agent 自动发现工具后可直接说:"抓取 https://example.com/products 的所有产品名称和价格,输出 JSON"。
架构一览
Agent → MCP/REST/CLI → Chameleon 门面
├─ SmartRouter(L0-L6 升级链 + 策略记忆)
├─ Pipeline(清洗 → Markdown → 提取 → 校验)
└─ DeepCrawler + Scheduler(策略遍历/限速/断点)测试
uv run pytest # 229 个测试(含 Playwright 真实渲染、反爬模拟站对抗)
uv run ruff check src tests
uv run mypy src边界与声明
反爬对抗是持续的军备竞赛,本工具不保证对任何站点必然成功:
持续演进的验证码(滑块/设备指纹/风控评分)可能击败任何自动化手段
内置升级链全程透明可观测——结果中记录实际使用的引擎与升级层级,Agent 可据此判断可信度
住宅代理与打码服务为第三方付费依赖,请按预算使用
仅在获得授权的范围内使用:遵守 robots.txt、站点服务条款、当地法律(GDPR/个保法)与限速要求。滥用造成的后果由使用者自行承担
文档
开源
Star History
Maintenance
Resources
Unclaimed servers have limited discoverability.
Looking for Admin?
If you are the server author, to access and configure the admin panel.
Related MCP Servers
AlicenseAqualityBmaintenanceA local MCP server that lets AI agents bypass bot detection, geo-restrictions, and JavaScript rendering challenges when scraping the web, backed by ScraperAPI's services285MIT- Flicense-qualityDmaintenanceMCP server for crawling and extracting data from web pages using Selenium with CSS/XPath selectors. Supports 17 tools including smart extraction, pagination, infinite scroll, and screenshots.3
- Alicense-qualityDmaintenanceAn MCP server that gives AI tools a reliable way to fetch content from the web, handling anti-bot protection and JavaScript-rendered pages.497MIT
- Alicense-qualityDmaintenanceMCP server for web scraping and browser automation, enabling AI agents to extract clean, token-efficient content from web pages.1MIT
Related MCP Connectors
Official Octoparse MCP server for template discovery, cloud tasks, and structured data export.
One MCP for 160+ live web-data APIs — clean JSON from sites that block scrapers.
Free remote MCP server for fetching public web pages through a rotating proxy pool.
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/xiabai2008/chameleon'
If you have feedback or need assistance with the MCP directory API, please join our Discord server