browser-relay

Drives the user's real, logged-in Chrome from AI agent sessions without ever handing another browser client the user's cookies (which gets sessions invalidated). An MV3 extension lives inside real Chrome and executes commands sent over a WebSocket from a local relay daemon.

Architecture

Claude session -> MCP (HTTP :9277/mcp) -> relay daemon -> WebSocket -> MV3 extension -> real Chrome tabs

extension/ - the Chrome MV3 extension.
server/relay.js - the relay daemon: MCP streamable-HTTP server + WebSocket server. Node >= 18, single dependency (ws).
deploy/ai.wecko.browser-relay.plist - launchd LaunchAgent template (macOS); install.sh fills in your node and repo paths.
install.sh - installs deps + the LaunchAgent and prints the two remaining manual steps.
test/ - MCP smoke test (e2e.sh), a direct tool-call helper (call.js), and a mock extension for testing the relay without Chrome.

The extension connects out to ws://127.0.0.1:9277/ws as a client (it does not run a server itself), sends a hello on open, and answers every command the relay sends with exactly one ok:true/ok:false reply.

Install

/bin/bash install.sh

This copies the LaunchAgent, (re)starts it, and waits for http://127.0.0.1:9277/health to answer. It then prints two steps you do by hand:

claude mcp add --transport http browserx http://127.0.0.1:9277/mcp -s user

Load the extension: chrome://extensions -> enable Developer mode -> Load unpacked -> select extension/.

Tools (relay commands)

command	args	result
`list_tabs`	`{}`	`{tabs:[...]}`
`new_tab`	`{url?, active?}`	`{id,url}`
`close_tab`	`{tab_id}`	`{closed:true}`
`activate_tab`	`{tab_id}`	`{id,title,url}`
`navigate`	`{url, tab_id?}`	`{id,url,title}`
`back`	`{tab_id?}`	`{id,url}`
`forward`	`{tab_id?}`	`{id,url}`
`snapshot`	`{tab_id?, max_elements?, include_text?}`	`{tab_id,title,url,frames,elements,truncated?,text_excerpt?}`
`click`	`{ref, tab_id?}`	`{clicked:ref}`
`type_text`	`{ref, text, submit?, clear?}`	`{typed:ref}`
`evaluate`	`{js, tab_id?, all_frames?}`	`{result, truncated?}` (capped ~6KB)
`screenshot`	`{tab_id?, format?}`	`{mime,data_base64,width?,height?}` (1280px max, jpeg default)
`wait_for`	`{text, tab_id?, timeout_ms?}`	`{found:true,waited_ms}`

tab_id is optional everywhere it appears - it falls back to the active tab of the last focused normal window.

snapshot is the main way an agent finds things to act on: it walks every frame of the page (iframes included) for interactive elements, tags each one with a stable data-bx-id="fN:eM" ref (per-frame prefix), and returns a compact list plus frame metadata. click, type_text and wait_for search all frames, so admin apps rendered in iframes (OVH, Shopify) work directly without evaluate fallbacks.

Cost guardrails (v1.1.0)

snapshot defaults to 150 elements per frame (hard cap 250) and the text_excerpt is opt-in via include_text.
evaluate output is capped at ~6KB (strings truncated, arrays/objects limited) and prefers compact JSON.
screenshot is downscaled to max 1280px, jpeg quality 75 by default.
click fires exactly one native el.click() (v1 fired 2 click events).
wait_for polls every 1.5s across all frames.

The relay enforces a backstop cap on every tool result, so no single call can inflate the model context.

Known limitations

Synthetic input is not trusted input. click and type_text dispatch real DOM events, but they are not OS-level input, so some sites (payment iframes, bot-detection-heavy forms) may reject or flag them.
Screenshots require the tab to be active. chrome.tabs.captureVisibleTab only captures the active tab of a window; screenshot briefly activates the target tab if it isn't already, then restores the previous one.
evaluate can be blocked by a page's CSP. Strict script-src policies can reject the injected eval; when that happens the error is surfaced verbatim rather than swallowed. A future version should switch to chrome.userScripts.execute in the USER_SCRIPT world, which is exempt from page CSP.

Changelog

v1.1.0 - cost/token optimizations: smaller snapshots, capped evaluate output, downscaled jpeg screenshots, single-click fix, iframe support for snapshot/click/type/wait.
v1.0.0 - initial release.
The MV3 service worker gets killed by Chrome. It is normal for background.js to be torn down and revived repeatedly; the bx-keepalive alarm (every ~24s) and the reconnect-on-close logic are what keep the WebSocket connection coming back. A short gap in connectivity right after Chrome starts or the SW is revived is expected.
Google (and similar) may challenge with reCAPTCHA or a security check. If that happens, the agent must stop and ask the human to clear it in the real browser window rather than trying to script around it.
Same-origin iframes are not walked by snapshot in v1.

Troubleshooting

curl 127.0.0.1:9277/health - confirms the relay daemon is up.
tail -f relay.log - relay daemon stdout/stderr (from the LaunchAgent).
Reload the extension from chrome://extensions if it stops responding.
Inspect the service worker console: chrome://extensions -> browserx relay -> "service worker" link, to see WebSocket connect/reconnect logs and any errors from injected scripts.