ProBrowser
by momadhuynh04
README.md
<div align="center">
# ProBrowser
### Real-Browser Automation MCP Server for AI Agents — Anti-Detection
**29 tools · CDP attach to your real Opera/Chrome · Human-like input · DOM + Vision hybrid**
[Features](#-features) • [Pipeline](#-pipeline) • [Quick Start](#-quick-start) • [Usage](#-usage) • [Configuration](#-configuration) • [Architecture](#-architecture) • [Project Structure](#-project-structure) • [Anti-Detection](#-anti-detection)
🌐 Language: **English** | [Tiếng Việt](#tính-năng)
</div>
---
> [!WARNING]
> **Dev & tested only on Arch Linux**
> kernel `Linux 7.1.5-arch1-2` · DE: **KDE Plasma**
> Other distros/desktops are untested — YMMV.
---
An **MCP Server** that lets AI agents (Kilo, Claude Code, …) control the user's **real browser** through the Model Context Protocol. No headless mode, no throwaway profiles — the agent attaches over **CDP** to the running Opera/Chrome with all sessions, cookies and logins intact, then drives it with human-like timing and a DOM-first / vision-fallback strategy.
## ✨ Features
| Feature | Description |
|---------|-------------|
| 🔌 **Real Browser Control** | `connect_over_cdp` attaches to the running browser on port 9222 — sessions, cookies, extensions preserved. Never launches headless or fresh profiles unless asked. |
| 🧰 **29 MCP Tools** | Navigation, tabs, click/type/hover/keyboard, scroll, drag & drop, DOM/text extraction, JS eval, 3 screenshot variants, CAPTCHA toolkit, system detection. |
| 🕒 **Human-Like Input** | Gaussian-distributed inter-key delays (45–75 WPM), natural mousedown↔mouseup gaps, cubic ease-out scrolling — layered as timing middleware, zero JS injection. |
| 🔍 **Hybrid DOM + Vision** | Structured DOM extraction (`get_dom`, `find`, `get_text`) first; screenshots + coordinate clicks as fallback when DOM is insufficient. |
| 🎯 **Position Validation** | Before any coordinate click: annotated screenshot with red X/Y axes crossing exactly at origin O=(x,y) + element probe, warnings (`pointer-events:none`, disabled, iframe). |
| 🧩 **CAPTCHA Toolkit** | Auto-detects reCAPTCHA/hCaptcha/Turnstile/Cloudflare after navigation; image-grid solving via agent vision (`captcha_grid` → pick tiles → `solve_captcha`); optional 2Captcha API auto-solve. |
| 🗂️ **Tab Management** | List / switch / open / close tabs — multi-tab workflows on one real browser. |
| 🚨 **Actionable Errors** | Every failure returns structured `{error_code, error, recoverable, suggestion}` — `TIMEOUT`, `SELECTOR_NOT_FOUND`, `CDP_CONNECTION_LOST`, `NETWORK_ERROR`… |
| 📚 **Self-Documenting** | `browse_help(topic=...)` ships 11 in-server topics; `browse_list_tools` mirrors the full catalog at runtime. |
| ✅ **Tested** | 87 unit tests (fake pages, no browser needed) + real-browser integration suite (`pytest -m integration`). |
## 🏭 Pipeline
```
┌──────────┐ ┌────────────┐ ┌─────────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ CONNECT │ → │ ORIENT │ → │ SEE │ → │ PLAN │ → │ ACT │ → │ VERIFY │
│ check_ │ │ navigate + │ │ screenshot │ │ selector │ │ click / │ │ re-read │
│ browser │ │ wait idle │ │ get_dom │ │ or coords│ │ type │ │ state │
└──────────┘ └────────────┘ └─────────────┘ └──────────┘ └──────────┘ └──────────┘
│ │
▼ not running ▼ DOM miss
┌──────────────┐ ┌──────────────────────────┐
│ open_browser │ │ validate_position(x, y) │
│ (real profile│ │ → annotated O=(x,y) shot │
│ + CDP flag) │ │ → click_position(x, y) │
└──────────────┘ └──────────────────────────┘
```
**Tool execution flow** (example `browse_click`):
```
agent call → pre-click Gaussian delay (~80ms)
→ PhantomWright Actionability Engine (visible? enabled? stable? unobscured?)
→ CDP Input.dispatchMouseEvent (trusted gesture, NOT element.click())
→ post-click pause (~150ms)
→ result JSON (+ optional verification screenshot)
```
Auto CAPTCHA/ad-overlay scan runs after every `browse_navigate` / `browse_wait`.
## 🚀 Quick Start
### Prerequisites
- **Arch Linux** (dev/test platform) — kernel `7.1.5-arch1-2`, KDE Plasma
- **Python 3.11+** — check with `python --version`
- **A Chromium-based browser** — Opera (recommended), Chrome, Chromium or Brave
- **An MCP client** — Kilo CLI, Claude Code, or anything that speaks MCP stdio
### Installation
```bash
# 1. Clone
git clone https://github.com/YOUR_USER/probrowser.git
cd probrowser
# 2. Setup (venv + deps + verify)
./setup.sh
# ...or manually:
python -m venv .venv
.venv/bin/pip install -r requirements.txt
```
### Register with your MCP client
`kilo.json` (project root):
```json
{
"mcp": {
"probrowser": {
"type": "local",
"command": ["/absolute/path/to/probrowser/.venv/bin/python", "-m", "src"],
"enabled": true,
"timeout": 120000
}
}
}
```
Claude Code users: same server via `.mcp.json`. Restart the client so the 29 `browse_*` tools load.
### First Run
```bash
# Option A — start Opera yourself with CDP enabled:
opera --remote-debugging-port=9222
# Option B — let the agent do it (real profile, cookies intact):
browse_check_browser() # → not connected?
browse_open_browser(browser="opera")
```
Then just talk to your agent: *"open YouTube and play One of These Nights"* — it will connect, navigate, click, and verify playback by itself.
## 💻 Usage
### Tool Groups
| Group | Tools |
|-------|-------|
| **Navigation & Tabs** | `browse_navigate` · `browse_wait` · `browse_list_tabs` · `browse_switch_tab` · `browse_open_tab` · `browse_close_tab` |
| **Interaction** | `browse_click` · `browse_validate_position` · `browse_click_position` · `browse_type` · `browse_hover` · `browse_keyboard` · `browse_scroll` · `browse_drag_drop` |
| **Extraction** | `browse_get_dom` · `browse_find` · `browse_get_text` · `browse_eval` |
| **Screenshots** | `browse_screenshot` · `browse_screenshot_viewport` · `browse_screenshot_save` |
| **CAPTCHA** | `browse_check_captcha` · `browse_captcha_grid` · `browse_solve_captcha` |
| **System** | `browse_detect_browsers` · `browse_check_browser` · `browse_open_browser` |
| **Meta** | `browse_help` · `browse_list_tools` |
Full parameter reference: [`docs/tools-reference.md`](docs/tools-reference.md) or `browse_help(topic="tools-reference")` at runtime.
### Example Flow
```pytb
→ browse_check_browser()
← { connected: false } # nothing on :9222 yet
→ browse_open_browser(browser="opera")
← { launched: true, profile: "~/.config/opera", cdp_url: "...:9222" }
→ browse_navigate(url="https://youtube.com/...")
← { success: true, title: "..." } # auto CAPTCHA scan ran here
→ browse_get_dom(search="One of These Nights")
← { count: 10, elements: [{type: "link", selector: "a[aria-label=...]"}] }
→ browse_click(selector="a[aria-label=...]")
← { success: true }
→ browse_eval(expression="() => {const v=document.querySelector('video'); return {paused: v.paused, time: v.currentTime}}")
← { paused: false, time: 3.1 } # verified: audio is playing 🎵
```
### Standalone Server
```bash
PROBROWSER_CDP_URL=http://localhost:9222 .venv/bin/python -m src
```
## ⚙️ Configuration
All runtime configuration is environment-variable driven:
| Variable | Default | Description |
|----------|---------|-------------|
| `PROBROWSER_CDP_URL` | `http://localhost:9222` | CDP endpoint to attach on startup |
| `PROBROWSER_TOOL_TIMEOUT_MS` | `120000` | Hard cap per tool call (shared event loop protection) |
| `PROBROWSER_CAPTCHA_ENABLED` | `1` | Master switch for CAPTCHA subsystem |
| `PROBROWSER_CAPTCHA_AUTO_DETECT` | `1` | Auto-scan for CAPTCHAs/ads after navigation |
| `CAPTCHA_API_KEY` | — | 2Captcha API key (text-challenge auto-solve; grid solving is agent-vision based) |
| `PROBROWSER_CAPTCHA_MAX_WAIT_MS` | `60000` | Max wait for API-assisted solve |
MCP client-side settings (`timeout`, `enabled`) live in `kilo.json` — kept out of git on purpose (see `.gitignore`).
## 🏛️ Architecture
```
┌──────────────────────────────────────────────────────────┐
│ AI Agent (Kilo / Claude Code) │
│ plan → browse_* tool call → evaluate → repeat │
└───────────────────────┬──────────────────────────────────┘
│ MCP Protocol (stdio, JSON-RPC)
▼
┌──────────────────────────────────────────────────────────┐
│ ProBrowser MCP Server (Python) │
│ tool registry (29) → action coordinator → humanize.py │
│ (Gaussian delays · WPM typing · scroll easing) │
│ analyzer/: dom_parser · hybrid_strategy · captcha │
└───────────────────────┬──────────────────────────────────┘
│ CDP — Input.dispatch* events only
▼
┌──────────────────────────────────────────────────────────┐
│ User's Real Browser (Opera/Chrome, port 9222) │
│ sessions · cookies · logins preserved │
└──────────────────────────────────────────────────────────┘
```
### Key design decisions
| Decision | Rationale |
|----------|-----------|
| **All Python** | PhantomWright + `mcp` SDK are both Python — one language, one process, zero IPC |
| **PhantomWright over OS input** | No ydotool/xdotool/KWin hacks; CDP-level events work under Wayland and never block user input |
| **CDP input over JS injection** | `Input.dispatchMouseEvent` is a trusted gesture; `element.click()` is detectable and fragile |
| **`connect_over_cdp` over launching browsers** | The user's session IS the product — no re-login, no lost cookies |
| **Agent owns vision** | Server extracts geometry/screenshots; the agent's own vision model decides coordinates |
## 📁 Project Structure
```
probrowser/
├── main entry
│ ├── src/__main__.py # python -m src → stdio MCP loop
│ └── src/server.py # registry (29 tools) + dispatch + timeouts + captcha hooks
├── core
│ ├── src/browser/connection.py # connect_over_cdp, session persistence, page recovery
│ ├── src/browser/tab_manager.py # tab discovery / switching
│ ├── src/humanize.py # Gaussian timing middleware
│ ├── src/errors.py # classify_exception → structured error codes
│ ├── src/config.py # env-driven configuration
│ └── src/logging.py # per-action latency/success logging
├── tools (src/tools/, 29 modules)
│ ├── navigate · wait · list_tabs · switch_tab · open_tab · close_tab
│ ├── click · validate_position · click_position · type · hover · keyboard
│ ├── scroll · drag_drop
│ ├── get_dom · find · get_text · eval_js
│ ├── screenshot · screenshot_viewport · screenshot_save
│ ├── check_captcha · captcha_grid · solve_captcha
│ └── detect_browsers · check_browser · open_browser · help · list_tools
├── analyzer (src/analyzer/)
│ ├── dom_parser.py # DOM → simplified JSON (interactive + visible only)
│ ├── hybrid_strategy.py # DOM-first, vision fallback
│ └── captcha.py # CAPTCHA/ad-overlay detection
├── captcha (src/captcha/)
│ ├── grid.py # grid geometry + cropped screenshot extraction
│ └── solver.py # tile clicking + verify flow
├── docs/ # 11 topics served via browse_help()
├── tests/ # 87 unit tests + integration suite
├── AGENTS.md / CLAUDE.md # operating guide injected into agents
└── dev-phase-mcp-server.md # dev-phase principles
```
## 🛡️ Anti-Detection
### Browser Evasion Stack
| Layer | Implementation | Detail |
|-------|----------------|--------|
| Engine | PhantomWright (patched Playwright driver) | `navigator.webdriver` hidden at driver level |
| Patches | Stealth patches before any page JS runs | `window.chrome`, plugins, WebGL vendor, media codecs |
| Input | CDP `Input.dispatch*` trusted gestures | Not `element.click()`, not OS-level simulation |
| Session | Real user profile + real IP + extensions | Nothing about the fingerprint changes |
| Timing | Humanize middleware on every action | Gaussian delays, natural WPM, eased scroll |
### What still works
Your logins, cookies, extension state, IP reputation, canvas/WebGL/audio fingerprints — everything stays native because the browser is *yours*.
### Limitations
Aggressive anti-bot services (Cloudflare Turnstile, hCaptcha behavioral analysis) may still trip on IP reputation. Fallback path: screenshots + `validate_position` → `click_position`, plus the CAPTCHA toolkit.
## 🧪 Testing
```bash
# Unit suite — fake pages, no browser required (87 tests)
.venv/bin/python -m pytest tests/ -q
# Integration suite — needs Opera running with CDP
.venv/bin/python -m pytest tests/test_integration.py -m integration
```
## Tính năng
| Tính năng | Mô tả |
|-----------|-------|
| 🌐 Điều khiển trình duyệt thật | Gắn vào Opera/Chrome đang chạy qua CDP — giữ nguyên session, cookie, đăng nhập |
| 🧰 29 tool MCP | Điều hướng, tab, click/gõ/hover/bàn phím, scroll, kéo-thả, trích xuất DOM/text, JS, ảnh chụp, CAPTCHA |
| 🕒 Giả lập người thật | Delay Gauss giữa các phím (45–75 WPM), nhịp click tự nhiên, scroll ease-out |
| 🔍 Hybrid DOM + Vision | Ưu tiên DOM có cấu trúc; thiếu mới chuyển sang ảnh chụp + tọa độ |
| 🎯 Xác thực tọa độ | Ảnh annotate trục X/Y cắt nhau đúng tại O=(x,y) trước khi click tọa độ |
| 🧩 Bộ xử lý CAPTCHA | Tự phát hiện sau điều hướng; giải grid bằng vision của agent; auto-solve qua 2Captcha |
| 🚨 Lỗi có hành động | Mỗi lỗi trả về `{error_code, suggestion}` — agent biết phải làm gì tiếp |
| ✅ Được kiểm thử | 87 test unit + bộ integration trên browser thật |
### Cài đặt nhanh
```bash
git clone https://github.com/YOUR_USER/probrowser.git && cd probrowser
./setup.sh # venv + dependencies + verify
opera --remote-debugging-port=9222 # hoặc để agent tự mở
```
Khai báo server trong `kilo.json` (xem [Quick Start](#-quick-start)), khởi động lại client, xong.
---
<div align="center">
Made with 🎵 on Arch Linux · [MIT License](LICENSE)
</div>
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo issues