# Umbra

Drive your own signed-in Chrome from an AI agent, one session at a time, without handing the agent your cookies.

Umbra is two pieces that pair on a shared key you generate: a Chrome extension (Manifest V3) that is the only process allowed to call Chrome, and a local MCP server that gives any MCP client a browser tool list. They talk only over an authenticated WebSocket on 127.0.0.1. Nothing leaves the machine.

## Why this exists

Remote-debugging a browser gives an agent everything at once: every tab, every cookie jar, every profile. Umbra takes the opposite position. The agent gets a tab group it created, the tabs inside it, and nothing else. Ask it to read a tab it does not own and the extension refuses before Chrome is ever touched.

Two tools sit outside that boundary on purpose, and both are listed as such in `docs/permissions.md`: `browser_find_tabs` and `browser_find_groups` report the title and URL of tabs a session does not own, which is how you hand one over, and `browser_cleanup_groups` matches tab groups by title across the whole profile so it can clear groups left behind by a session that is gone. Give `browser_cleanup_groups` a title prefix your own groups do not share, or run it with `dryRun: true` first.

That boundary is what makes it usable against a browser you are already logged into. The agent can read a dashboard you are signed into, fill a form, export a CSV, and close its own tabs when it finishes, while your other windows stay untouched and unreadable.

The project is deliberately boring:

- real Chrome, real profile, real signed-in state, chosen by you
- many concurrent agent sessions in one browser, each isolated to its own tab group
- per-session tab ownership enforced on every action
- no cookie export, no token extraction, no CAPTCHA solving, no generic background fetch
- loopback-only transport with an HMAC handshake in both directions

## Install

Five steps, about five minutes, on macOS or Linux with Node.js 20+ and Chrome 121+. The full walkthrough with every option lives in `docs/install.md`. If you installed the extension from the Chrome Web Store, skip step 2; everything else is the same.

1. Get the code and install the companion:

   ```bash
   git clone https://github.com/RobertJLora/umbra
   cd umbra/mcp-server
   npm install
   ```

2. Open `chrome://extensions`, turn on Developer mode, click **Load unpacked**, and pick the `extension/` folder inside the clone from step 1. The options page opens by itself; if it does not, click Details on the Umbra card, then Extension options.

3. Run `node cli.js pair`. It prints the key and a ready-made client config block. Paste the key into the options page and click **Save And Reconnect**; paste the config block into your MCP client's config.

4. Click **Grant Site Access** on the options page. Chrome asks once; without it, page reads and screenshots fail.

5. Restart your MCP client. The options page status dot turns green within about fifteen seconds and the tools appear. If it does not, `node cli.js doctor` says why.

Dependencies live in `mcp-server/`, not at the repository root, so `npm install` at the root installs nothing. Once step 1 has run, `npm test`, `npm run doctor`, and `npm run release:check` all work from the root. The public checkout carries no optional local plugins.

## Tool surface

**Session and tabs**
`browser_create_tab`, `browser_list_tabs`, `browser_switch_tab`, `browser_close_tab`, `browser_close_session_tabs`, `browser_freeze_session_tabs`, `browser_group_tabs`, `browser_cleanup_groups`, `browser_mark_debug_group`, `browser_tabs_context`, `browser_get_session_status`, `browser_get_bridge_pressure`

**Adopting tabs you already opened**
`browser_find_tabs`, `browser_adopt_tab`, `browser_find_groups`, `browser_adopt_group`

**Navigation**
`browser_navigate`, `browser_navigate_back`, `browser_navigate_forward`, `browser_wait`, `browser_resize`

**Reading**
`browser_get_page_content`, `browser_read_page`, `browser_read_interactive`, `browser_find`, `browser_get_technical_snapshot`, `browser_screenshot`, `browser_console_messages`, `browser_read_network_requests`

**Interaction**
`browser_click`, `browser_click_text`, `browser_type`, `browser_fill`, `browser_form_input`, `browser_select_option`, `browser_hover`, `browser_drag`, `browser_press_key`, `browser_shortcut`, `browser_scroll`, `browser_file_upload`, `browser_upload_image`, `browser_cursor`

**Recording**
`browser_gif`

**Composites that save round trips**
`browser_batch`, `browser_wait_click_read`, `browser_navigate_wait_read`, `browser_click_wait_selector_read`

**Escape hatches**
`browser_javascript`, `browser_run_page_action`, `browser_wait_for_download`

`browser_reload_extension` exists for unpacked developer installs only. It is advertised when `UMBRA_ALLOW_EXTENSION_RELOAD=1`, and store installs refuse it. The options-page Reload button covers the same workflow without exposing a cross-session tool.

The list above is the whole surface of every published build. A checkout can carry optional local page-recipe plugins, which are not part of any published build: a module in `mcp-server/plugins/` paired with a page recipe in `extension/recipes/`. Both folders are untracked and unpublished, and a plugin adds its own tools and its own `browser_run_page_action` values to the list only in the install that holds it.

Notes worth knowing before you call these:

- `browser_get_page_content` defaults to text-only and supports selector scoping plus a `maxChars` cap. Pass `includeImages: true` only when you need the visible-image inventory.
- `browser_batch` runs a bounded create, navigate, wait, read, click, fill, press, scroll, close workflow in one MCP call. Child params can reference earlier results with `{"$ref":"prev.tabId"}` for the last successful step, `{"$ref":"0.tabId"}` for a step by index, or `{"$ref":"create.tabId"}` when that earlier call set `label: "create"`.
- `browser_read_interactive` returns a compact list of visible controls with short-lived refs tied to the current DOM version. `browser_click`, `browser_fill`, `browser_scroll`, and `browser_screenshot` accept those refs; a stale ref returns an error telling the caller to read again.
- `browser_get_bridge_pressure` reports one session's pressure: its owned tab count and a sample of those tabs, connected listener counts, and content-agent queue depth. It also reaps ownership records for tabs that no longer exist, so it is not purely read-only.
- `browser_freeze_session_tabs` discards owned inactive tabs with `chrome.tabs.discard` to release renderer memory. It defaults to `dryRun: true` and never targets a tab another session owns.
- `browser_run_page_action` runs predefined, named page actions and returns JSON-safe output. It is not an arbitrary script tool; `browser_javascript` is, and it routes through the debugger on the owned tab.
- The three composites default to leaving Chrome in the background and report which it was as `activated`. A child result reporting `active: true` means the tab is the active tab of its own window, not that Chrome came forward; `activated` and a screenshot preflight's `windowFocused` are the fields that answer the focus question.
- `browser_find` treats `selector` as a constraint rather than a single search root: every element it matches, and everything inside those elements, is searched. Its accessibility walk now filters for find rather than walking everything, so an existing caller gets a different candidate set than it did before.
- `browser_wait` takes a selector, `urlContains`, `urlChanged`, or a plain `durationMs` sleep. Use a URL predicate after a submit or a click, because a selector that is already on the un-navigated page matches instantly and hides the failure. `durationMs` on its own never touches the page, and combined with a predicate it settles first and then checks.
- `browser_click` takes `button`, `clickCount` 1 to 3, and `modifiers` for ctrl, shift, alt and meta. Omit all three and the call is the single unmodified left click it was before they existed.
- `browser_drag` drags between two points, refs, or selectors. It fires the pointer and mouse sequence every custom slider and sortable list reads, and adds the HTML5 drag family with one shared `DataTransfer` when the source carries `draggable="true"`, which is what a native drop target needs.
- `browser_upload_image` has two modes. Naming a file input with `ref` or `selector` hands Chrome the path and has no size limit; dropping at `x` and `y` sends the bytes through the server and is capped at 700 KB, so prefer the input mode whenever the page has one.
- `browser_press_key` takes a space-separated sequence such as `Tab Tab Enter` plus a `repeat` of 1 to 100, capped at 400 dispatches for one call. A single key with no repeat still dispatches one chord, but an unmodified Enter now also performs the browser default action, which is the "Enter is emulated" entry under Known limitations: it clicks the form's default submit button, or submits the form when there is none. Pass `defaultAction: false` to get the raw key dispatch and nothing else. The same true-by-default `defaultAction` applies to `browser_type` with `submit: true` and to `browser_shortcut`.
- `browser_scroll` takes `direction` with `amount` in roughly 100-pixel clicks, and `atX` with `atY` to scroll the inner pane under that point instead of the window. `x` and `y` keep their pixel-delta meaning.
- `browser_get_page_content` `mode: "article"` scores the page for body-text density and returns the winning block without the navigation, sidebars and related-link rails that `mode: "main"` leaves in. When the content agent served the read it reports the node it chose as `articleRootSelector`, so a disagreement is one `selector` read away; the one-shot fallback, which runs when the agent is unreachable, omits that field, and `contentAgent.fallback` says which path answered.
- Clicks, typing, scrolling and hovers draw a cursor inside the page: a pointer glides to the target, then a ripple, caret or chevron marks the action. It is painted in the tab, so a background tab stays in the background, and `browser_screenshot` clears the pointer and every ripple, caret and chevron before it captures. The glide is started rather than waited on, so the animation overlaps the real dispatch instead of delaying it. `browser_cursor` turns the drawing off or on for one session; the options page holds the install-wide default. The empty `<umbra-cursor-layer>` host is present on every driven page either way, and is stripped from HTML reads.
- `browser_read_network_requests` logs one owned tab's HTTP requests: URL, method, resource type, status, MIME type and timing. Filter with `urlPattern` and `types`, and pass `stop: true` when you are done. No request or response body is captured, and no header is returned. URLs keep their query strings, up to 600 characters, so a page that signs its URLs or carries a token in a query string puts that value in the log.
- `browser_tabs_context` returns `url` exactly as Chrome reports it. Pass `urlMaxLength` with a positive number to collapse the query and fragment to a marker and cap the length; those rows carry `urlTruncated: true` and their `url` is no longer navigable.
- `browser_gif` cannot run as a `browser_batch` child with `action: "export"`: inside a batch neither `outputPath` guard applies, so the encoded animation would come back inline. Call it directly.
- `browser_gif` records one owned tab: `start`, then the work, then `stop` and `export` with an `outputPath`. Frames come from the same background capture `browser_screenshot silent: true` uses, so recording never pulls the tab forward. Interval frames arrive at `fps`, four a second by default, and each click, drag, keystroke and scroll adds a frame before and after the action. The export writes the file and returns only metadata, never the animation itself. A recording stops on its own after three minutes.

## How it fits together

1. Your MCP client talks to the local server over stdio.
2. The server registers a session, either directly on a loopback bridge listener or through the Rust broker.
3. The Chrome extension's offscreen document holds the WebSocket and keeps it alive across service worker churn.
4. The extension authenticates every connection with an HMAC challenge over the shared key plus per-session nonces.
5. The background service worker assigns each session its own tab group and checks ownership before every Chrome call.

Two transports exist. The pure-Node bridge is what you get after a normal install. The Rust broker is what the launcher uses only after you build it: one extension WebSocket, many lightweight MCP shims registering sessions behind it over a local Unix socket, with the broker owning routing, auth, pressure counters, and request cleanup. Legacy mode gives each session its own loopback listener and is one setting away with `UMBRA_BROKER_MODE=legacy`. Either way the extension is the only thing that touches a Chrome API.

## Concurrency and ownership

- One Chrome profile hosts many sessions at once.
- Each session gets one session id, one named cyan Chrome tab group, and its own view of the browser.
- Opening, navigating, and DOM interaction default to inactive tabs, so routine work never pulls Chrome to the foreground. `browser_switch_tab` only retargets the session by default. Pass `activate: true` to select a tab in its window without OS focus; pass `allowForeground: true` with it only when Robert explicitly allows Chrome to come forward (FOREGROUND RULE).
- Umbra routes new session tabs into an existing Chrome window as inactive tabs. It prefers an unfocused window. It will not call chrome.windows.create unless allowForeground is true, because creating a window steals OS focus on macOS. If the only window is the one in use, it attaches an inactive tab there rather than opening a second window.
- Navigation is scheme-limited at the extension boundary: `http:`, `https:`, `file:`, and `about:blank` are allowed, and risky schemes such as `javascript:` and `data:` are rejected before Chrome sees them.
- At task completion the agent should call `browser_close_session_tabs`, which closes the whole owned group. It closes a whole window only when every tab in that window belongs to the session, so unowned blank tabs survive.
- Clean server shutdown runs the same cleanup by default. Set `UMBRA_KEEP_TABS_OPEN=1` or `UMBRA_CLOSE_ON_SHUTDOWN=0` when a run should leave tabs open for inspection.
- The default port range is `47821-47852`, which is wide enough that ordinary multi-agent work never runs out of room. The extension clamps a configured port to `1024-65535`; set the same range on both sides.

## What Umbra will not do

- dump or sync cookies
- extract tokens
- expose storage read and write as tools
- fetch in the background on a page's behalf
- solve CAPTCHAs
- touch bookmarks, history, or the clipboard
- use native messaging
- auto-update, auto-pull, or auto-install anything

## Known limitations

- Screenshots default to silent background capture (`Page.captureScreenshot` via debugger). Pass `silent: false` or `activate: true` only when a visible capture is required. Silent capture may show Chrome’s automation banner on the owned tab; it does not steal OS focus.
- Site access is an optional permission, requested from the Grant Site Access button on the options page rather than at install. Until it is granted, page reads, clicks, and screenshots fail with a message naming that button. `docs/permissions.md` justifies every permission the extension declares.
- Download completion is detected by watching the filesystem, because the extension does not request Chrome's `downloads` permission. Point `UMBRA_DOWNLOAD_DIR` at your browser's download folder if you moved it.
- A recording is cancelled when the bridge is disabled or the shared key is cleared. Both close the offscreen document, and that document is where the frames are held, so an in-flight recording dies with it. Chrome allows one offscreen document per extension, so there is no second place to hold them.
- The request log starts on the first read rather than being always on, so that first call usually returns nothing: act on the page and read again. Capture needs a debugger attachment, which shows Chrome's automation banner, and Umbra will not hold one on every tab for traffic nobody asked to see. Logging ends on `stop: true`, when the tab closes, when the session disconnects, and after five idle minutes.
- `browser_read_interactive` is intentionally compact. Umbra does not expose a full accessibility-tree dump.
- Generic text clicks can hit the wrong control on dense app UIs such as search pagination. Use `browser_read_interactive` with refs, or `browser_run_page_action` with `inspect_controls` then `click_control`, instead of guessing.
- Enter is emulated. Umbra clicks a form's default submit button, or submits the form when it has none, and skips that when the page handled Enter itself. Pass `defaultAction: false` for raw key dispatch. Chrome only performs implicit form submission for a real keypress, so a dispatched Enter on its own reaches page listeners and does nothing else.
- A right click is synthetic. The page gets a `contextmenu` event and renders its own menu if it has one, but Chrome's native context menu does not open, because no event a page can receive opens it. Use `browser_shortcut` when the native menu is the point.
- Back navigation can fall through to the page's own history. Chrome hides history entries left behind by clicks that had no user gesture, so the tab-level back call reports an empty stack on a tab that plainly has one. Umbra retries in the page and reports which path moved it as `via`.
- Changing the port range needs both sides to reload: restart the MCP client so new server processes inherit the environment, and reload the unpacked extension so persisted extension storage is normalized.

## Layout

- `extension/` - MV3 extension: background worker, offscreen bridge, content agent, options page, popup
- `extension/recipes/` - optional site-specific page recipes, injected on demand and absent from the published package
- `extension/vendor/` - third-party code shipped as is, currently the MIT-licensed GIF encoder the recorder uses
- `mcp-server/` - stdio MCP server, loopback bridge, Rust broker shim client, and the local development harness
- `rust-broker/` - Tokio broker runtime that multiplexes sessions over one extension WebSocket
- `tests/` - auth, ownership, session isolation, extension lifecycle, and packaging coverage
- `scripts/` - isolated Chrome test-profile launcher and smoke wrappers
- `launchd/` - template for the optional macOS job that keeps the broker running
- `docs/` - install, architecture, permissions, and smoke-test notes

## Documentation

- `docs/install.md` - setup from clone to a connected session, plus every environment variable
- `docs/branding-and-icons.md` - mark lineage, the four icon surfaces, dark-mode icon switching, and the store update flow
- `docs/architecture.md` - components, flow, and the reasoning behind the offscreen and background split
- `docs/permissions.md` - each Chrome permission with its risk and its mitigation
- `docs/smoke-test.md` - automated and manual verification paths
- `docs/performance/performance-work.md` - what the performance pass changed and what it measured
- `MCP_PROTOCOL.md` - the wire protocol between extension and server
- `THREAT_MODEL.md` - assets, trust boundaries, attackers, and mitigations
- `SECURITY_REVIEW.md` - review stance, the keep and remove matrix, and upstream audit findings
- `rust-broker/README.md` - broker scope and how to run it
- `rust-broker/LEGACY_FALLBACK.md` - rollback triggers and the cutover shape

Read `THREAT_MODEL.md` and `SECURITY_REVIEW.md` before pointing this at a browser that holds anything you care about.

## Release / Chrome Web Store

Same-cycle CWS when main ships a version: see `store/RELEASE_PROCESS.md`. Ash packages; Rex submits.