Skip to main content
Glama

Deputy

We taught a website to describe itself to an agent — without its cooperation.

Add to Chrome · Deputy Pro · launch film · measurements · architecture

https://github.com/user-attachments/assets/ffd3eeb2-2ed4-4bb8-83d7-c809d50e8fff

This repository is the free, open-source version of Deputy: you build it from source, and forms as tools need Chromium's WebMCP flag. The full Deputy is on the Chrome Web Store. Deputy Pro (hosted models, browser_look): razin.ai/deputy.

Measured on a busy page. An admin page with 400 orders: filter them and count the matches. Claude Sonnet in Claude Code, the same prompt, median of three runs, every run passed. Measured with the full Deputy, not this open-source build.

Claude + Playwright MCP

Claude + Deputy

turns

10

3

Claude tokens

245,250

72,632

3.4× fewer

total cost

$0.189

$0.072

2.6× cheaper

Method: docs/measurements.md §0.

Deputy is an agent that lives in your browser. Any MCP client — Claude Code, Codex, Cursor, your own agent — stops driving the browser itself and asks Deputy instead. It hands back a typed API for whatever page you are on, then performs the call in your real, logged-in session.

The page never enters the calling agent's context. No screenshot, no DOM, no accessibility tree.


The idea

A <form> already knows everything about itself. It knows a field is a required date, that a dropdown has exactly three options, that a number must fall between 1 and 12. We throw all of that away when we render to pixels, and then pay a language model to infer it back from a picture.

There is a W3C standard that fixes this — WebMCP, shipping in Chromium 152 — which lets a site declare its forms as agent-callable tools. Almost nobody has adopted it.

So Deputy adopts it on their behalf. It writes four HTML attributes onto the page's own form, and then Chromium itself generates the JSON Schema and executes the submission:

<form toolname="book_a_table" tooldescription="…">
  <input name="party_size" type="number" min="1" max="12" toolparamdescription="Party size">
// what the agent receives — generated by the browser, not by us
{ "party_size": { "type": "number", "minimum": 1, "maximum": 12 },
  "date":       { "type": "string", "format": "date" },
  "seating":    { "type": "string", "enum": ["indoor", "outdoor", "bar"] } }

We wrote no schema generator and no form-filler. The browser already had both.

It also never submits for you. On any form that changes something, Deputy withholds WebMCP's auto-submit attribute, so Chromium fills the fields, focuses the submit button and waits for a person to press it. That is consent enforced by the browser, not by a check we could forget.

Related MCP server: chrome-in-harness

CopilotKit: we meet on the same standard

CopilotKit v2 ships a WebMCPRegistry that publishes an app's frontend tools to document.modelContext — the same WebMCP registry Deputy grafts onto everything else. So the integration needs no glue at all:

flowchart LR
  ck["CopilotKit v2 app<br/>useFrontendTool(...)"] --> mc["document.modelContext"]
  dp["Deputy<br/>grafts forms and ARIA widgets"] --> mc
  mc --> bc["browser_capabilities"] --> agent["any MCP agent"]

Both directions work, and both are real:

  • CopilotKit → any agent. An app's frontend tools are registered with live handlers, so Deputy surfaces them to Claude Code fully executable — not a description, the actual action. An app built for its own in-app copilot becomes usable by any agent, with no change to the app.

  • Deputy → CopilotKit. Deputy's grafted tools land in the same document.modelContext that CopilotKit reads through getWebMCPModelContext(). A CopilotKit copilot gains tools for forms its developers never wired up.

For CopilotKit v1, which predates the WebMCP registry, Deputy falls back to reading the actions manifest as the app POSTs it to /api/copilotkit — read-only, request passed through untouched. Capability without cooperation, same as the rest of Deputy.

Pages with no <form>

Most of the interesting web has no <form> element. Google Forms, React apps, anything that builds a form out of divs. Deputy synthesizes the tool itself there — including the widgets that are not form controls at all:

on screen

actually in the DOM

Deputy emits

multiple choice

role="radiogroup" → role="radio"

enum

checkboxes

role="checkbox"

boolean

dropdown

role="listbox" → role="option"

enum

paragraph answer

contenteditable

string

Measured on a Google-Forms-shaped page: 1 field captured before, 5 of 5 after. It also queries across open shadow roots, because a page built from web components looks empty otherwise.

Either way — real form or not — the agent makes one typed call. That equivalence is the point: falling back to "click this, then type that" hands the per-step cost straight back to the caller, which is the thing Deputy exists to remove.

What it costs

Filling a 7-field booking form. Same prompt, same data, both through claude -p. Both succeeded.

Claude + Playwright MCP

Claude + Deputy

turns

9

6

Claude tokens

216,888

133,366

1.63× fewer

Deputy's own model

—

1 call · $0.000014

total cost

$0.2730

$0.1861

1.47× cheaper

wall clock

26.1 s

17.2 s

1.52× faster

Per observation, on the same Wikipedia page:

representation

~tokens

full accessibility tree

215,495

screenshot (PNG)

57,134

trimmed a11y snapshot (what Playwright MCP sends)

14,170

Deputy's typed schemas

115

The baseline was given the accessibility-snapshot path, not just screenshots, so the comparison is deliberately conservative. Method and both tasks: docs/measurements.md.

The finding worth reading twice

browser_capabilities first returned tool names only, to keep the payload small. The agent had to probe for parameter names: 246,003 tokens. Returning full schemas: 155,774. Adding more still — current values, plus every button and link with a clickable ref: 133,366, 6 turns.

~5× more data made the task 1.8× cheaper. Every probe an agent is forced to make costs more than the context that would have prevented it.

Tools the agent sees

Ten, and they never change as you browse, so the caller's context stays flat.

tool

browser_capabilities

the page's contract: typed schemas, current values, clickable actions

browser_invoke

run a tool; fills the real form, and leaves a state-changing submit to you

browser_click

click an action from browser_capabilities, by its ref

browser_fill

set one field by ref, on pages whose form could not become a typed tool

browser_ask

ask a question about a page; Deputy reads it and answers (~40 tokens)

browser_read

the page as text, when you need the content to reason over

browser_navigate

open a URL and report what appeared

delegate_goal

hand over a goal and let Deputy choose

task_status

check a delegated goal that is still running or was interrupted

answer_task

resume a delegated goal that stopped for a person: a confirmation, or a sign-in

Long pages are windowed, never silently truncated: Wikipedia's 263 links come back as "Showing 1-25 of 263" with actionQuery to filter and actionOffset to page.

Two agents, two models

Deputy is not Claude with extra steps. It runs qwen/qwen3.7-flash via OpenRouter, because reading a page and choosing a form is not work that needs a frontier model. Claude thinks; Deputy browses.

With no API key it falls back to a keyword planner and still works — the demo never depends on a second key being up.

Run it

bun install
cp .env.example .env       # add OPENROUTER_API_KEY (optional — it degrades without one)
bun run dev                # daemon + a Chromium carrying the extension

Then point any MCP client at http://127.0.0.1:7331/mcp — one streamable-HTTP endpoint, loopback only, no auth. Deputy is not Claude-specific; it is the same ten tools whatever is calling.

Claude Code

claude mcp add --transport http deputy http://127.0.0.1:7331/mcp

Codex

codex mcp add deputy --url http://127.0.0.1:7331/mcp

Cursor — ~/.cursor/mcp.json, or .cursor/mcp.json for one project

{ "mcpServers": { "deputy": { "url": "http://127.0.0.1:7331/mcp" } } }

VS Code / GitHub Copilot — .vscode/mcp.json

{ "servers": { "deputy": { "type": "http", "url": "http://127.0.0.1:7331/mcp" } } }

Windsurf, Cline, Zed, Goose, Continue — the same mcpServers block as Cursor. Windsurf spells the key serverUrl; Cline and Zed use url.

Claude Desktop, or any host that still speaks only stdio — bridge it:

{ "mcpServers": { "deputy": {
  "command": "npx", "args": ["-y", "mcp-remote", "http://127.0.0.1:7331/mcp"] } } }

Your own agent — POST JSON-RPC at the endpoint, or hand the URL to an MCP SDK's streamable-HTTP client. scripts/sweep.ts is a working client in about ten lines, and doubles as a smoke test.

bun test              # 88 unit tests, never launches a browser
bun run test:browser  # 10 integration tests against real Chromium
bun run demo          # serve the example pages used in the video
bun scripts/sweep.ts  # point Deputy at 8 real sites and see what it makes of each

How it's built

flowchart LR
  client["Any MCP client<br/>Claude Code · Cursor · …"] -- MCP --> deputyd
  subgraph deputyd ["deputyd (Bun)"]
    direction TB
    registry["registry: tabs → tools"]
    tasks["tasks: A2A-shaped states"]
    planner["planner: qwen (optional)"]
  end
  deputyd -- WebSocket --> extension
  subgraph extension ["extension (MV3)"]
    direction TB
    graft["graft: annotate real forms"]
    synth["synthesize: everything else"]
    content["content: getTools / executeTool"]
  end
  extension --> webmcp["Chromium's own WebMCP"]

The daemon exists because an MV3 service worker dies after 30 s idle and cannot listen on a port. The content script runs in the ISOLATED world, so Deputy never needs chrome.debugger and you never see the "extension is debugging this browser" banner.

Design notes and invariants: ARCHITECTURE.md.

Prior art

PinchTab solves an adjacent problem well — browser control for agents without screenshots, via an accessibility tree with element refs. Its fill <ref> and text commands inspired browser_fill and browser_read.

The difference: PinchTab gives an agent a better map of the page. Deputy gives it a typed API. A seven-field form is one validated call rather than seven ref-targeted actions, and Chromium performs the submission natively because the form was retrofitted into a real WebMCP tool.

What's real, and what isn't

Tested and running: the retrofit engine, ARIA-widget synthesis, shadow-DOM traversal, in-app copilot manifest capture, tool naming, the tab registry, the task lifecycle, the extension, the daemon, and every number above. 85 unit + 10 integration tests, the latter against live Chromium.

Not built: the A2A wire format. Deputy's task states are A2A's vocabulary — input_required, auth_required, because a browser agent hits login walls constantly — but no Anthropic product speaks A2A, so MCP got the time. Multi-page navigation memory is designed in ARCHITECTURE.md §8 and not built.

Known limits: needs chrome://flags/#enable-webmcp-testing (WebMCP is in origin trial through Chrome 156). A live Google Form is untested — the fixture replicating its DOM passes, but Google's real page is heavily obfuscated. Capabilities behind an interaction are invisible until you take it: MDN's search lives in a modal, so Deputy offers the button, not the field.

One thing worth knowing

Every bug that mattered was found by running it. executeTool in Chromium 152 wants its arguments as a JSON string; passing an object stringifies to [object Object] and fails. That cost one full run — 19 turns, 452,165 tokens — of Claude patiently working around a broken tool. The rest are in docs/measurements.md.

License

PolyForm Shield 1.0.0. Use it, change it and share it for anything, including at work, except to provide a product that competes with Deputy. Re-publishing it counts as competing, even for free.

Versions up to and including commit 5d38eee were released under the MIT License.

Available Tools

27 tools
answer_taskAnswer a paused taskB

Resume a task that stopped because it needed a human — a confirmation, or a sign-in that has now been completed in the browser.

Inputs: answer (required), taskId (required).

browser_actAct on an elementA

Do one thing to an element: click, double_click, right_click, hover, focus, type (value; replaces unless append), press (keys like "Enter", "Tab", "Escape", "ArrowDown", "Control+a"; without a target, presses on the focused element), check, uncheck, select (value = option label or value, native or custom dropdown), clear, scroll_into_view. Typing is sent key by key so autocompletes react. The result reports any dialog or error the action caused. For whole forms, prefer browser_invoke with the typed tool.

Inputs: ref, keys, steps, tabId, +6 more.

browser_askAsk about a page instead of looking at itA
Read-only

Ask a question about what is on a browser tab. Deputy reads the page itself and returns a short answer. Use this instead of screenshotting or dumping the DOM — the page never enters your context, so a lookup costs a couple of hundred tokens rather than tens of thousands. Without a model, it returns the passages relevant to your question instead.

Inputs: question (required), tabId.

browser_capabilitiesWhat can the browser do right now?A
Read-only

The full contract of every open tab: each form as a typed tool with its JSON Schema (types, allowed values, min/max, required fields), what those fields currently hold, and every button and link with a ref you can click. This is what you read instead of a screenshot — it is complete, so you should not need to probe. Long pages return a window of their links: the reply always states the true total, and you can narrow with actionQuery or page with actionOffset.

Inputs: actionLimit, actionQuery, actionOffset.

browser_clickClick a button or linkB

Click something browser_capabilities listed under "actions", by its ref.

Inputs: ref (required), tabId.

browser_consoleConsole and errorsB
Read-only

Console messages and page errors: uncaught exceptions with stacks, unhandled rejections, failed resources, and console output from when an agent started driving the tab. Filter by levels (error, warn, info, log, debug). Browser-internal messages (CSP, CORS) are not visible without the debugger.

Inputs: clear, limit, tabId, levels.

browser_dialogAnswer dialogsA

Decide how alert/confirm/prompt dialogs are answered on a tab you are driving. By default they are dismissed (a confirm returns false) and reported in the action result. Call this BEFORE the action that opens the dialog: accept for the next dialog, persist to keep answering that way.

Inputs: accept (required), text, tabId, persist.

browser_downloadsDownloadsA
Read-only

Files the browser downloaded. "list" shows recent downloads; "wait" waits for the one just started (by a click) to finish; "read" returns a finished download by id (the newest if no id): text files as text, PDFs as extracted text, other files as their path, type and size.

Inputs: action (required), id, timeoutMs.

browser_dragDrag and dropC

Drag one element onto another (HTML5 drag-and-drop, or pointer-driven libraries such as kanban boards).

Inputs: tabId, toRef, fromRef, toSelector, +1 more.

browser_evalRun JavaScript in the pageA

Run JavaScript in the page and return its JSON-serialisable result. An expression returns its value; for statements, use return. Promises are awaited. Runs in the page's own world by default (sees its globals); world "isolated" keeps it apart. Needs Chrome's "Allow User Scripts" switch for Deputy; the error says how to turn it on.

Inputs: code (required), tabId, world.

browser_extractExtract dataA
Read-only

Turn a table, or a repeated list (search results, product cards, feed items), into JSON rows; or list the links. kind "auto" picks the biggest table, else the biggest repeated list. Scope it with a ref or selector. Pair with browser_scroll untilStable for infinite feeds.

Inputs: ref, kind, limit, tabId, +1 more.

browser_fillFill one field by refA

Set a single field to a value, addressed by the ref browser_capabilities gave it. Use this on pages Deputy could not graft into a typed tool — apps that build a form out of divs, or inputs sitting outside any . For a checkbox pass "true"/"false"; for a pass an option value or its visible text. When a typed tool exists, browser_invoke is better.

Inputs: ref (required), value (required), tabId.

browser_findFind elementsA
Read-only

Find elements by visible text, role (button, link, textbox, checkbox, combobox, option, heading, tab, menuitem, dialog, img, row, cell) or CSS selector. Returns refs you can pass to browser_act, browser_inspect or browser_screenshot, with label, state and value. Reaches inside forms, open and closed shadow roots, and elements off screen.

Inputs: role, text, exact, limit, +3 more.

browser_inspectInspect layout and CSSA
Read-only

With a ref or selector: the element's box model, position, computed CSS (or the properties you name), flex/grid layout, visibility, text contrast, overflow, and what is covering it. With no target: a layout audit of the whole page — horizontal overflow and its culprits, clipped text, broken images, missing alt text, small tap targets, low-contrast text, layout shift, fonts. Numbers instead of a screenshot, for CSS debugging.

Inputs: ref, tabId, selector, properties.

browser_invokeDo something on the pageB

Call one of the tools browser_capabilities listed, with arguments matching its inputSchema. This acts in the real logged-in browser: it fills and submits the actual form.

Inputs: tool (required), args, tabId.

browser_lookLook at the page (Pro)A
Read-only

A vision model looks at the page and answers in text: describe it, or answer your question about what is visible (layout, images, charts, whether something looks broken). Costs you a few hundred tokens instead of an image, and works in clients that cannot show images. Deputy Pro.

Inputs: tabId, question.

browser_navigateOpen a URL, or go back, forward or reloadA

Point a tab at a URL and wait for it to load, or move through its history with action "back" / "forward", or "reload" it. Deputy re-reads what the new page can do, so call browser_capabilities afterwards to see the tools that appeared.

Inputs: url, tabId, action.

browser_networkNetwork requestsA
Read-only

Requests the page made: method, URL, status, time, type, size, for fetch/XHR and every resource (documents, scripts, styles, images). failedOnly shows 4xx/5xx and network errors. includeBodies adds text/JSON response bodies (first 4 KB) captured while an agent drives the tab.

Inputs: clear, limit, tabId, filter, +2 more.

browser_readRead a page as textA
Read-only

Return the page as plain text for you to read yourself. Use browser_ask instead when you only need one fact: with a model (Deputy Pro or OPENROUTER_API_KEY) Deputy reads the page and answers in a sentence. Use this when you genuinely need the content to reason over.

Inputs: tabId, maxChars.

browser_resizeResize the viewportA

Set the page viewport to a width and height in CSS pixels, e.g. 375×812 to check a mobile layout. This resizes the user's real window: call with reset: true when done to put it back.

Inputs: reset, tabId, width, height.

browser_screenshotSee the pageA
Read-only

Take a screenshot and return the image to you. An image costs you roughly 1,000-1,600 tokens, so use it when you need to SEE something: visual or frontend work, layout and styling, charts, images, canvas. For facts on a page prefer browser_ask or browser_read; to act, prefer browser_capabilities. fullPage stitches the whole scrollable page, including anything off the right edge (costs more). ref or selector captures a single element whole, even one wider or taller than the viewport.

Inputs: ref, tabId, format, fullPage, +2 more.

browser_scrollScrollA

Scroll the page (or an element) down/up, by pixels, or to top/bottom. untilStable keeps scrolling to the bottom while an infinite feed keeps loading, and reports how much it grew. Finds the real scroll container in app layouts where the window itself does not scroll.

Inputs: by, to, ref, tabId, +4 more.

browser_tabsList, open, switch or close tabsA

Work with browser tabs. "list" shows every tab; "open" opens a URL in a new tab and makes it active; "select" makes a tab active (other tools act on the active tab by default); "close" closes one.

Inputs: action (required), url, tabId.

browser_uploadUpload filesA

Attach local files (absolute paths, 25 MB total) to a file input, or drop them onto a drop zone.

Inputs: paths (required), ref, tabId, selector.

browser_waitWait for the pageA

Wait until text appears, text disappears, or an element becomes visible — for pages that load content after the fact (search results, spinners, SPAs). With no condition, waits timeoutMs. Fails with a clear message on timeout.

Inputs: text, tabId, selector, textGone, +1 more.

delegate_goalDelegate a goal to the browserA

Hand the browser agent a goal in plain language and let it work. It knows the page, the session you are signed into, and what each form accepts. Prefer this over driving the browser yourself — you never need to see the page.

Inputs: goal (required), tabId.

task_statusCheck a delegated taskA
Read-only

Poll a task by id. Use when a delegation is still running or was interrupted.

Inputs: taskId (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.6/5.0

    Scored across 27 tools

    Disambiguation4/5

    Most tools have a clear niche, and descriptions explicitly resolve the main overlaps (browser_invoke vs browser_fill/browser_click, browser_read vs browser_ask vs browser_look vs browser_screenshot). However there is genuine redundancy in the action/read space—clicking can be done via browser_click, browser_act, or browser_invoke—so selection still requires care. The guidance text largely saves it, keeping it at mostly distinct.

    Naming Consistency4/5

    The 23 browser tools all follow a clean browser_<verb> pattern (browser_click, browser_fill, browser_navigate), which is highly predictable. The four task-level tools (task_status, delegate_goal, answer_task) break the pattern with mixed noun_verb/verb_noun ordering, a minor deviation rather than chaos.

    Tool Count3/5

    27 tools is heavy and pushes past the comfortable range, with arguably too many overlapping read/act primitives (read, ask, look, inspect, extract, capabilities). Each does cover a distinct browser capability, so it is defensible rather than padded, but the set is borderline oversized for the domain.

    Completeness5/5

    The surface covers the full browser-driving lifecycle: navigation and tabs, form discovery and submission, raw actions, waits, scrolling, uploads/downloads, dialogs, screenshots, console, network, and JS eval. The delegation flow (delegate_goal, task_status, answer_task) closes the human-in-the-loop gap, leaving no obvious dead ends.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      A
      quality
      B
      maintenance
      Enables AI agents to see and control the user's real Chrome/Brave/Edge profile over MCP, so they can read pages, click, type, take screenshots, audit layouts, debug CSS, and scrape paginated or infinite-scroll data. Because it drives the normal browser via the DevTools protocol with real input events, it works on modern JavaScript apps and on sites where the user is logged in.
      25
      1
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables any MCP client to drive the user's real Chrome profile through a Chrome extension bridge, with tools for navigation, accessibility snapshots, ref-based interactions, console/network reading, screenshots, tab management, and restricted script evaluation.
      1
      MIT
    • A
      license
      Not graded
      quality
      A
      maintenance
      Enables an MCP client's LLM to drive a live browser by stating one outcome at a time, while TypeSafe System One picks the elements, actions and values and judges whether each step is done, blocked or irreversible; it also exposes tools for snapshots, targeted page reading, screenshots and calling a page's own WebMCP tools.
      1,464 npm
      MIT
    • A
      license
      Not graded
      quality
      A
      maintenance
      Lets an MCP client drive a real browser by stating one outcome per step, with a local model deciding which element, action, and value to use and pausing before irreversible actions. Also enables direct page reading, snapshots, screenshots, and calling a page's own WebMCP tools.
      1,464 npm
      MIT