Skip to main content
Glama
Fire162

browserpilot-mcp

by Fire162
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

šŸš€ BrowserPilot

Control your local desktop browser directly from remote VPS AI Agents via Model Context Protocol (MCP)

TypeScript Chrome Extension Model Context Protocol License

šŸ’” Why BrowserPilot?

When building autonomous AI agents on a remote VPS, interacting with modern websites is challenging:

  • Traditional headless browsers (like standard Puppeteer/Playwright) get blocked by Cloudflare, reCAPTCHA, and bot-detection systems.

  • Authenticating into your personal accounts (Google, GitHub, banking, dashboards) on a headless VPS requires syncing cookies and session tokens.

  • Remote debugging ports (--remote-debugging-port=9222) trigger Chrome's prominent yellow "Browser is being controlled by automated software" banner.

BrowserPilot solves this by establishing a secure, persistent outbound WebSocket bridge directly from your local Chrome/Brave/Edge browser to your VPS MCP server. Your VPS AI agent can interact with your real, authenticated browser tabs using realistic DOM events while keeping the connection ultra-lightweight and invisible.

Related MCP server: MCP Chrome Bridge

šŸ›ļø Architecture

sequenceDiagram
    autonumber
    actor User as You (Local PC)
    participant Ext as Chrome Extension (MV3)
    participant Relay as Offscreen Document (Persistent WS)
    participant MCP as VPS MCP Server
    actor Agent as AI Agent (Claude/Cursor/AGY)

    User->>Ext: Loads Extension & Enters VPS Endpoint
    Ext->>Relay: Initializes Background Offscreen Relay
    Relay->>MCP: Outbound WebSocket Connect (ws://<your-vps-ip>:8765?token=...)
    MCP-->>Relay: Auth Verified (200 OK)
    
    rect rgb(30, 41, 59)
        note right of Agent: AI Agent executes browser action
        Agent->>MCP: Call Tool: browser_click({ selector: "#submit" })
        MCP->>Relay: Send JSON Command (ID: cmd_101)
        Relay->>Ext: Dispatch to Content Script
        Ext->>Ext: Highlight element & dispatch native mouse events
        Ext-->>Relay: Action Succeeded
        Relay-->>MCP: Return Result (ID: cmd_101)
        MCP-->>Agent: Tool Response: "Clicked #submit successfully"
    end

šŸ› ļø MCP Tools Reference

BrowserPilot exposes 12 specialized tools directly to any MCP-compatible AI agent:

MCP Tool

Description

Key Parameters

browser_status

Checks if local browser extension is connected and reports latency.

None

browser_list_tabs

Lists all open tabs across your browser windows with titles & URLs.

None

browser_navigate

Navigates the current tab (or opens a new tab) to a given URL.

url, newTab?, tabId?

browser_switch_tab

Switches focus and brings a specific tab to the foreground.

tabId

browser_close_tab

Closes a specific tab.

tabId?

browser_read_page

Extracts readable text, clean markdown, or interactive element catalog.

format? (markdown, interactive_elements, text, html), maxLength?, tabId?

browser_click

Clicks an element by CSS selector or human-readable text label.

selector?, text?, tabId?

browser_type

Types into an input/textarea with realistic input events.

selector, text, clear?, pressEnter?

browser_press_key

Dispatches keyboard events (Enter, Escape, Tab, ArrowDown).

key, tabId?

browser_scroll

Scrolls the page in any direction or scrolls an element into view.

direction? (up, down, top, bottom), amount?, selector?

browser_take_screenshot

Captures the active viewport and returns base64 image data to the agent.

tabId?

browser_evaluate

Runs custom JavaScript expression in the page and returns the result.

script, tabId?

šŸ“¦ Project Structure

browserpilot/
ā”œā”€ā”€ mcp-server/              # Model Context Protocol Server (VPS side)
ā”‚   ā”œā”€ā”€ src/
ā”‚   ā”‚   ā”œā”€ā”€ index.ts         # Stdio MCP Server & lifecycle entry
ā”‚   ā”‚   ā”œā”€ā”€ websocket-hub.ts # WebSocket server & command dispatcher
ā”‚   ā”‚   ā”œā”€ā”€ tools.ts         # MCP tool definitions & schema validation
ā”‚   ā”‚   ā””ā”€ā”€ types.ts         # Protocol message interfaces
ā”‚   ā”œā”€ā”€ test/                # Automated bridge integration tests
ā”‚   ā””ā”€ā”€ package.json
ā”‚
ā”œā”€ā”€ extension/               # Manifest V3 Chrome Extension (Local side)
ā”‚   ā”œā”€ā”€ manifest.json        # Extension configuration & permissions
ā”‚   ā”œā”€ā”€ background.js        # Service worker & tab router
ā”‚   ā”œā”€ā”€ offscreen.html/js    # Offscreen document (unbreakable WebSocket keep-alive)
ā”‚   ā”œā”€ā”€ content.js           # In-page DOM engine & element highlighter
ā”‚   ā”œā”€ā”€ popup.html/css/js    # Settings popup UI
ā”‚   ā””ā”€ā”€ icons/               # Extension icons
ā”‚
ā””ā”€ā”€ package.json             # Root pnpm workspace

šŸš€ Quickstart Guide

Step 1: Start the MCP Server on your VPS

  1. Clone or copy the repository to your VPS:

    cd /root/browserpilot
pnpm install

  2. Configure environment variables in mcp-server/.env:

    WS_PORT=8765
SECRET_TOKEN=my-secure-browserpilot-token

  3. Build and test the MCP server:

    pnpm build
pnpm --filter browserpilot-mcp exec tsx test/test-bridge.ts
TIP

šŸŒ Best Practice: Zero-Config Deployment with Fire PM Tunnels

Instead of manually opening firewall ports or wrestling with SSL certificates, you can supervise BrowserPilot 24/7 and expose an encrypted HTTPS / WSS tunnel using Fire PM ā€” the native Linux process supervisor & tunnel ecosystem.

  1. Install Fire PM (if not already installed): Visit the Fire PM Repository or run the installer:

    git clone https://github.com/Fire-Package/fire-pm.git /root/fire-pm
cd /root/fire-pm && sudo ./install.sh

  2. Start BrowserPilot as a Persistent System Service:

    fire start /root/browserpilot/mcp-server/dist/index.js --name browserpilot --env WS_PORT=8770 --env SECRET_TOKEN=my-secure-token

  3. Open a Public Secure Tunnel (Automatic SSL/WSS):

    fire tunnel open 8770
# Output: āœ” Custom Tunnel established for localhost:8770
#         šŸ”— URL: https://<hash>-tunnel.yourdomain.com

  4. Connect from Chrome Extension: In the extension popup, enter:

    • VPS WebSocket Endpoint: wss://<hash>-tunnel.yourdomain.com

    • Secret Token: my-secure-token

  5. Manage Your Tunnel & Service:

    fire list              # View service health and memory usage
fire tunnel list       # View active tunnels and uptime
fire logs browserpilot # Tail live service logs
NOTE

šŸ›”ļø Manual Firewall Configuration (Alternative)

If you are not using Fire PM tunnels and are connecting directly over raw TCP, make sure port 8770 (or your custom WS_PORT) is open:

1. Ubuntu / Debian (UFW)

sudo ufw allow 8770/tcp comment "BrowserPilot WebSocket"
sudo ufw reload

2. RHEL / CentOS / AlmaLinux / Rocky (firewalld)

sudo firewall-cmd --permanent --add-port=8770/tcp
sudo firewall-cmd --reload

3. Raw iptables

sudo iptables -A INPUT -p tcp --dport 8770 -j ACCEPT

Step 2: Install the Chrome Extension on your Local Computer

  1. Copy or download the extension/ folder from your VPS to your local PC.

  2. In your local browser (Chrome, Brave, Edge):

    • Navigate to chrome://extensions

    • Enable Developer mode in the top-right toggle.

    • Click Load unpacked and select the extension folder.

  3. Click the BrowserPilot icon in your browser toolbar:

    • VPS WebSocket Endpoint: ws://<your-vps-ip>:8765 (or wss://tunnel.yourdomain.com)

    • Secret Token: my-secure-browserpilot-token

    • Click Connect.

  4. The badge will turn šŸŸ¢ ON and status will display "Connected to VPS".

Step 3: Connect your AI Agent to the MCP Server

Add BrowserPilot to your agent's MCP configuration:

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "browserpilot": {
      "command": "node",
      "args": ["/path/to/browserpilot/mcp-server/dist/index.js"],
      "env": {
        "WS_PORT": "8765",
        "SECRET_TOKEN": "my-secure-browserpilot-token"
      }
    }
  }
}

Antigravity CLI (agy) or Custom Agent

{
  "mcpServers": {
    "browserpilot": {
      "command": "node",
      "args": ["/root/browserpilot/mcp-server/dist/index.js"],
      "env": {
        "WS_PORT": "8765",
        "SECRET_TOKEN": "my-secure-browserpilot-token"
      }
    }
  }
}

šŸ”’ Security & Privacy

IMPORTANT

The WebSocket bridge allows arbitrary command execution inside your browser session. Always protect your connection:

  • Pre-Shared Secret: Set a strong SECRET_TOKEN in your environment.

  • Encryption: When running over public networks, route through an encrypted tunnel (Cloudflare Tunnel, Tailscale, or Nginx with Let's Encrypt wss://).

  • Visual Feedback: When an AI agent clicks or interacts with elements, BrowserPilot highlights them with green/blue halos in real-time so you always see what the agent is doing.

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • Chrome Extension MCP Bridge

    ivoglent
    F
    license
    Not graded
    quality
    D
    maintenance
    Enables AI agents to control the Google Chrome browser through a Node.js WebSocket bridge and a dedicated browser extension. It provides tools for capturing screenshots, executing JavaScript, managing tabs, and extracting page content via the MCP protocol.
    2
    -

  • MCP Chrome Bridge

    wizardofweb125-lab
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI agents to control Chrome browser actions like navigation, clicking, form filling, screenshots, and console/network logging via an MCP server and Chrome extension.
    821 npm
    MIT

  • Tabryn

    idugeni
    A
    license
    Not graded
    quality
    A
    maintenance
    Connects AI agents to your Chrome browser via MCP, enabling real-time control of existing tabs, sessions, and application state for development workflows.
    MIT