Linux CUA

Linux desktop toolkit for computer-use agents: isolated X11 workspaces, a Rust X11/AT-SPI engine and an MCP surface.

Status: prototype. An exploratory, unreleased project: commands, interfaces and install steps can change without notice.

What it does

Agents get private, throwaway X11 desktops (agent-desktop), a computer-use engine that reads and drives their apps through AT-SPI and X11 without touching the owner's display, and model-facing surfaces (MCP and provider adapters). The engine is a clean-room implementation: no OpenAI binaries, JavaScript packages, or instruction text are included. The supplied research specification, public AT-SPI/X11 interfaces, and public provider API documentation informed the implementation.

Tested with a GTK3 fixture on a private Xvnc display. The MCP server and both provider adapters have exercised the real engine. Live Claude/GPT inference, Codex reference timings, XWayland, and broad application compatibility remain unverified. Native Wayland is not implemented.

One engine, three surfaces

• MCP JS: persistent QuickJS/WASM context in a separate restricted process. js {title,code,timeout_ms}, js_reset, turn_ended. Top-level await and bindings persist. No model filesystem, shell, module loader, or network API.

• MCP JSON: discrete window/AX/capture/action tools over the same daemon.

• Provider adapters: AnthropicComputerToolset for computer_toolset_20260801, OpenAIComputerTool for {type:"computer"}. Host application owns inference and human approval. See surface usage.

const app = await cua.getApp("xed");   // focused main window; or {appId}, {windowId}. Emits its state
// Use an element ID from that observation, not a guessed ID:
await app.setValue(observedFieldId, 'Grüße λ');
await app.click(observedButtonId);   // the cell output ends with the state after the last action
const png = await app.getScreenshot({emit: false});      // PNG bytes (Uint8Array)
await nodeRepl.emitImage(png);
nodeRepl.write('done');

The JS API mirrors the names and shapes of OpenAI's computer-use JavaScript API, because GPT models are trained on it: WindowInfo {id, app, title?}, AppInfo {id, displayName?, isRunning?, windows?}, cua.getState, listApps, listWindows, getApp({windowId}), cua.launchApp(name) (starts an installed application and returns its App once its window is up), cua.clipboard.write/read/release, cua.help() (prints the API description again, for a model that lost its context), and App methods getAXState, getScreenshot, getAXStateAndScreenshot (one engine request for both), click, doubleClick, rightClick, moveMouse/hover, moveMouseRelative, drag, scroll, typeText, pressKey, paste, setValue, performSecondaryAction. click, drag and scroll take {modifiers: "shift+ctrl"} (or an array) for held modifier keys; click and pressKey take {durationMs} to hold the button or key. Methods also accept GPT's single-object forms (click({element}), click({x, y}), typeText({text}), pressKey({key}), pressKey(["ctrl", "s"]), setValue({element, value}), scroll({x, y, scrollY}), ...); key names are case-insensitive (Ctrl+s, Enter).

For continuous canvas strokes, await app.drag({path: [[100,100], [180,100], [180,180], [100,100]]}) holds the left button through every point (2–256 integer window-local points). Use the app's pencil or brush for arbitrary outlines; shape tools interpret a drag according to their own rules. The engine checks the complete path before pressing, releases on interruption, and settles once after release. The original drag(from, to) remains available. Group predictable strokes in one JS cell and inspect a screenshot afterward.

An App covers all top-level windows of its process: dialogs (WM_TRANSIENT_FOR), modal windows and popups. cua.getApp("xed") picks the app's focused main window, names its other main windows, and emits the app's state, so no getAXState() is needed after it (one in the same cell before any action is not emitted again). An unknown name fails with the list of running app IDs, so listApps() is not needed first. Observations show the app's focused or modal window, plus a one-line list of its other windows. Element IDs stay valid across those windows, and actions on an element inside a dialog just work. Keys and text go to the app's focused window. Actions observe implicitly when needed, so one cell can open a dialog, fill it, confirm and verify.

setValue commits a spin button through AT-SPI Value when available, including range validation and numeric readback; changing only its editable text can leave the application using the old number. Other editable fields keep text insertion and readback.

typeText/paste insert through the focused AT-SPI EditableText. In a single-line field one trailing newline becomes Return after the insertion; a newline inside the text is refused with a hint. Without a focused EditableText: in a dialog, the text goes into its only text field, otherwise it is refused with input_refused (always in an alert or a dialog whose tree is unknown), so that text meant for another window cannot answer an unexpected "Save changes?"; in a main window, and only when that window is the one the model last saw, typeText types key events and paste pastes through the clipboard (Ctrl+V, so text the keyboard map cannot type still arrives), both read back where the app reports the focused text. cua.launchApp("drawing") starts an installed application by its desktop entry ID (org.gnome.Calculator), the ID's last part, its menu name or its program name, never a command line; terminal applications, the deny list and the hard denies apply, and the new app still needs the usual approval before it is read or controlled. cua.clipboard.write(text) takes the desktop's clipboard (served by the daemon on its own X connection until an app takes it over), read() returns its text or null, release() gives it up. Reading text that an app copied is a read of that app: it needs the app allowed or approved in the current turn, and a denied app's clipboard (or one owned by a client without a listed window) is never read. A mismatch from typeText or setValue, for example a decimal comma locale turning "125.50" into "12550", is reported as a warning with a hint and does not throw. Advice that saves a turn starts with hint: and is shown once per session the first time it reaches the model: apps with number input (spin buttons, an editable display) get the decimal-comma hint in their first observation when LC_NUMERIC uses one, and a file manager item view that exposes no items over AT-SPI (Thunar's icon view) suggests the detailed list view (Ctrl+2), whose rows are clickable table cells. getAXState({query: "..."}) returns only the matching elements with their ancestors and descendants. Wherever an element ID is accepted (click, setValue, performSecondaryAction, scroll, moveMouse), a locator {role, name, text, states} names the element instead: it is resolved against a fresh observation of the app's active window when the action runs, waits up to 1 s for a match, and fails with element_not_found, or ambiguous_element listing the candidates, unless exactly one element matches. Roles accept common spellings (push button, textbox, checkbox); name matches the accessible name exactly (case-insensitive) before partially, or as a RegExp; text matches name, description, text or value. app.find(locator, {timeoutMs}) returns the matches without emitting them. A keyboard shortcut (a Ctrl, Alt or Super chord other than the editing chords, or a function key) and a button click wait 150 ms of quiet instead of 35 ms, so the next action of a cell reaches the dialog they open. window_id exists only on the raw NDJSON side and in the discrete MCP tools. Observation and discovery calls emit their result into the cell output unless called with {emit: false}; nodeRepl.write(value) and await nodeRepl.emitImage(image) add output explicitly (console.log and display are aliases). getAXState returns the changes since the tree the model last received for that window (the last emitted observation), never since an internal observation in the same cell; {disableDiffing: true} returns the full tree. Coordinates are window-local pixels of the window of the last screenshot or tree model code received (frame_window on the raw side). If the app's active window changed since, for example because the previous click opened a dialog, coordinate input is refused with input_refused until the model looks again; if that window closed, it fails with window_not_found. scroll(...,{pixels:n}) is explicit; a numeric distance uses 500 pixels per page. Top-level const bindings are mutable across REPL cells. Rich clipboard formats are rejected; the clipboard carries plain text only.

The Rust daemon uses NDJSON stdin/stdout (see RPC contract). Actions serialize to keep observations and input ordered. AT-SPI reads inside a snapshot are pipelined. The daemon keeps a mirror of each app's objects: the bulk Cache.GetItems result plus everything fetched live since (dialog widgets and tree-view cells that atk-bridge never caches). An AT-SPI event names the objects it touches, and only those (with their children) are fetched again at the next observation; the mirror is rebuilt after 5 s as a backstop. The window's accessible frame, once mapped, is reused with a single call instead of enumerating the app's top-levels. Element IDs come from one counter for the whole session and are never reused, so an ID from an old observation can never resolve to a different element. When the host restarts the daemon (timeout, cancellation), it passes --first-element-id past the last ID it saw. IDs are window-bound and revalidated before actions. An observation's first call waits up to 1 s for a busy app (a file chooser populating after it opened), because an observation without a tree costs the model a turn; an app confirmed blocked inside an earlier accessibility action gets 300 ms. The tree walk after that has a 500 ms budget; inner call timeouts derive from that deadline. A Cache.GetItems that times out is not retried for 10 s and a deadline-bounded live walk returns a partial tree with a warning instead; the previous element map stays valid until a new snapshot succeeds. An app that does not answer is reported and later re-probed briefly (60 ms) rather than paying the full wait on every action. An action with observe: false returns the window it landed on and the app's windows without a tree: the MCP surface sends it for every action in a js cell and observes once when the cell ends, and the provider adapters send it because they capture a screenshot after the batch. screenshot: true on an observation or action attaches the window's image to the same result (getAXStateAndScreenshot is one request). Raw results include the full text; MCP results (JS and discrete tools) show the full tree or, when shorter, the changes since the tree the model last received for that window.

X11 SendEvent input targets a window without pointer warps, XTEST, or foreground activation. Coordinate input can also hit the window's own popups (menus, dropdowns, tooltips: override-redirect windows of the same X client); a point over a subwindow owned by another client (XEmbed or reparented windows) is refused. Held modifiers (modifiers) are pressed as key events to the window before the pointer events and carried in their state; hold_ms keeps a button or key down (physical input still aborts, and the release is always sent); mouse_move hovers (enter/leave and motion without a button; GTK shows tooltips about 0.5 s later). Semantic actions use AT-SPI. A window is redirected with Composite as soon as it is approved or first read, and the first read waits (up to 300 ms) for the client's repaint, so a fresh backing pixmap never shows pixels of overlapping windows; resizes are handled the same way. Capture reads the backing pixmap with MIT-SHM when supported, otherwise GetImage, and composites the client's mapped popups on top. It refuses an occlusion-sensitive capture if Composite fails. A virtual cursor overlay tracks coordinate actions. Settling has two phases: wait up to 200 ms for the first sign the app reacted (damage on the window or its popups, a popup mapping, a top-level window being created, or an AT-SPI event of the app; a GTK app building a file chooser shows nothing for 70-100 ms on an idle machine), then until that activity has been quiet for 35 ms, capped at 600 ms; while the app has reacted only with accessibility events and shown nothing yet (it is building a dialog, with pauses), the quiet interval is 150 ms. If nothing reacted, the observation carries a warning. Settling also waits (up to 1.5 s) for a top-level window the settle target's client is creating to map (windows already mapped when the wait starts are skipped), and after Return, Enter or Escape, a button click, or text ending in a newline inside a dialog, for that dialog to close: up to 1.5 s from the action, ending 350 ms after the dialog's last X or AT-SPI activity, or after 1 s if it showed none (the GTK file chooser opens a typed file 400-900 ms after Return with no activity before it unmaps, the longer on a loaded machine), so a dialog that stays open after a navigation (a file chooser entering a typed folder) costs about 0.45 s; a dialog that closes after a longer pause would cost the model a turn, which is why the waits err on the patient side. A dialog whose accessible frame is already gone when it is observed is closing: the observation waits up to 300 ms for it to unmap and shows the window behind it. So the next action in a cell reaches the dialog that just opened, or the window behind a dialog that just closed. This is a heuristic, not proof a network request completed.

Related MCP server: claude-workman

Install

Requirements: current stable Rust, Node ≥22.13, an explicitly owned X11 display with Composite, DAMAGE, XFIXES, XInput2 and X-Resource ≥1.2, and a private D-Bus/AT-SPI session. MIT-SHM is optional. Fixture tests additionally use Python3/PyGObject GTK3 and xdotool. There is no dependency on agent-desktop in the engine or surface.

git clone https://github.com/5omeOtherGuy/linux-computer-use
cd linux-computer-use
cargo build --release --locked
npm ci --prefix surface
cargo test --locked
npm test --prefix surface

Then link the desktop commands and skill from a release or dedicated installation checkout. Links follow that checkout, so installed tools should not point to a worktree being edited:

python3 scripts/install-desktop.py
# To migrate existing command/skill symlinks from a separate checkout:
python3 scripts/install-desktop.py --replace-links

The installer creates symlinks under ~/.local/bin and ~/.agents/skills/agent-desktop; it never replaces regular files or installs packages. The optional owner-facing panel remains in desktop/panel/. The MCP server and benchmark harness prefer the bundled CLI; CUA_AGENT_DESKTOP overrides that choice. The engine can still use another explicitly owned private X11 session independently.

With a custom CARGO_TARGET_DIR (for example on a larger disk) and CARGO_BUILD_JOBS=2 on small machines, set CUA_ENGINE to the built binary accordingly.

Usage

Bring your own private display/session. Never point this at another agent's display. Display 0 and the default owner session/a11y bus paths are refused. No implicit $DISPLAY fallback exists.

Example using the bundled agent-desktop commands after installation:

W=$(agent-desktop new --label 'Linux CUA' --geometry 1024x768 --a11y)
eval "$(agent-desktop env "$W")"
export CUA_DISPLAY="$DISPLAY"
export CUA_ENGINE="$PWD/target/release/linux-cua"
agent-desktop run "$W" -- /usr/bin/python3 "$PWD/bench/fixture.py"
node surface/bin/linux-cua-mcp.js
# After the client exits, close only this workspace:
agent-desktop close "$W"

Configure any MCP stdio client to launch node /absolute/path/surface/bin/linux-cua-mcp.js with CUA_ENGINE, CUA_DISPLAY, XAUTHORITY, the private DBUS_SESSION_BUS_ADDRESS, and optionally AT_SPI_BUS_ADDRESS. On a workstation with agent-desktop, CUA_WORKSPACE=new replaces those: the server creates a private workspace with accessibility (CUA_WORKSPACE_LABEL, CUA_WORKSPACE_GEOMETRY), owned by the server process and closed when it exits, and apps started with cua.launchApp run in it; CUA_WORKSPACE=NAME attaches to an existing workspace instead and leaves it open. The server uses desktop/bin/agent-desktop from this checkout, falling back to PATH or ~/.local/bin for a standalone surface installation (MCP clients such as Codex start servers with a reduced environment); CUA_AGENT_DESKTOP sets its path explicitly. The client must support form elicitation, or apps must be explicitly allowed by the trusted operator. Denials always win. Nothing is installed into your Amp/Claude configuration automatically.

Live demos on your own desktop (explicit opt-in)

The daemon refuses display :0 unless it is started with an explicit command-line opt-in. Environment variables cannot enable it, so it is never inherited from the MCP surface (which also refuses :0 itself). Only a trusted host that you start, such as a demo bridge, can pass it:

CUA_DISPLAY=:0 linux-cua --live-desktop --allow-app gnome-calculator [--allow-app APP_ID ...] \
  (--pid PID | --allow-exe /usr/bin/gnome-calculator) [...]

• --allow-app (required, repeatable) is the exact, case-insensitive WM_CLASS app ID. Windows of other apps are never listed, read or controlled.

• At least one --pid or --allow-exe (repeatable) is required, because any client can set any WM_CLASS. A window is admitted only if its process is one of the --pids or its /proc/<pid>/exe is one of the --allow-exe paths (canonicalized; checked when the window is listed).

• Window-to-process binding uses the X server's own record of the client that created each window (X-Resource QueryClientIds, LocalClientPID), never the client-writable _NET_WM_PID. Windows whose client PID the server cannot report (remote clients) are not listed.

• The per-app approval, deny lists and hard denies still apply. The AT-SPI bus check is unchanged: the owner's default session and accessibility buses are refused, so semantic actions work only for apps started on a private bus (as the demo bridge does); coordinate input and capture work either way.

• The former CUA_AUTHORIZED_WINDOW_IDS/CUA_AUTHORIZED_PID variables are rejected at startup.

Safety boundaries and configuration

• CUA_ALLOW_APPS / CUA_DENY_APPS: comma-separated exact, case-insensitive WM_CLASS app IDs. Apps not on the allow list require MCP approval before any window content is read or controlled. Approval covers the app's process, so its dialogs too. The user chooses the scope in the approval form: this turn, this session (until the MCP server exits), or always. "Always" decisions are stored in CUA_APPROVALS_FILE (default $XDG_CONFIG_HOME/linux-cua/approvals.json, {"always_allowed": [{"app_id": "Xed", "executable": "/usr/bin/xed"}]}). An entry matches only that app ID together with that executable; turn and session approvals are keyed the same way, and the daemon additionally binds an approval to the process start time, so a reused PID does not inherit it. A malformed or old-format file is reported on stderr and treated as empty, never overwritten. The host reads the file once at startup, so edits (by the operator or by any process running as the same user) apply to later sessions only. Only the trusted host writes it, after an explicit user choice. Listing window metadata does not require approval.

• launch_app starts only installed desktop entries (Type=Application, shown in menus), by name, with Exec field codes expanded to nothing; the model never supplies a command line or arguments. The controlled apps run as the same user, so entries and programs that user can write are never launched: an entry and its program must sit where every directory and symlink on the way belongs to another user (normally root) and is not group- or world-writable. Entries in ~/.local/share/applications and programs in ~/.local/bin are therefore skipped, though a user entry (including a Hidden or NoDisplay one) still hides the system entry with the same ID. The same rule applies to the command of an env NAME=VALUE COMMAND launcher, to a script's #! interpreter (including #!/usr/bin/env NAME) and to Path=; apps start in / unless Path= says otherwise, and their PATH keeps only trusted directories. What an app runs later, and its own configuration and plugins in the user's home, are outside this rule. Entries with Terminal=true, on the deny list or matching the hard denies (by desktop ID, name, program or WM class) are refused, and the started app's windows need the usual approval. It is unavailable on a live desktop, as is the clipboard.

• Terminal/password-manager/security-prompt identifiers and executable names are hard-denied, even if allowed or approved. The executable is resolved from the X-Resource-verified client PID; a window whose process or executable cannot be established is denied, as are unknown app identities. WM_CLASS can still be chosen freely by a client; this is not an isolation boundary against malicious native clients. The same holds for the clipboard: its owner is identified by the window it names, and X lets a client name another client's window. Embedded terminal widgets and browser-hosted password managers are not comprehensively classified. Use a private desktop with only trusted fixture/apps, not your personal desktop or secrets.

• XI2 key/button/motion/touch input latches an abort. The engine checks before input, during semantic operations and long X11 actions, and after settling. A trusted turn_ended resets it and approvals. Display-wide abort is intentionally more conservative than window-only abort.

• Confirmation-policy text is original and advisory; it is not a transaction-risk classifier. A separate reviewer model is not implemented. Ask before external or irreversible actions.

• Cancelling or timing out kills the issuing daemon and stops further dispatch. Events already sent to an app cannot be recalled; the outcome may be partial or unknown. Never automatically replay an interrupted action. Turn end also cancels active JS cells and queued old-turn requests.

• Raw NDJSON is a trusted-host API, including approve_app; never expose it directly to model code or over an unauthenticated network socket. The model's only bridge goes through host validation and approval.

• Bus addresses are checked after percent-decoding and path canonicalization; implicit (runtime=, dir=, tmpdir=, autolaunch), non-unix and abstract-socket addresses are refused (CUA_ALLOW_ABSTRACT_BUS=1 accepts abstract sockets).

• The daemon's own 20 s request timeout is best effort because X11 waits block its thread. The real request deadline is the trusted host, which kills the daemon (SIGKILL) on timeout or cancellation.

Compatibility

Last verified 2026-10-18: the display-free checks in scripts/gate on Ubuntu 24.04 with Rust 1.99.1, Node 24 and the system Python 3. The X11 behaviour was last measured on 2026-10-16 on an XFCE/X11 workstation with private Xvnc displays (see measurements).

• Native Wayland needs a portal/libei/PipeWire backend behind the existing RPC. XWayland should expose the X11 path but has not been tested.

• Some apps ignore SendEvent. Keyboard input currently targets the top-level window; child-window focus routing and toolkit coverage need expansion. Verify state after every action.

• typeText/paste require text. typeText never touches the clipboard; its key-event fallback can type only characters on the display's keyboard map. paste without a text field to insert into replaces the clipboard with the text and presses Ctrl+V, so it reaches only apps bound to that shortcut.

• Element clicks are pointer clicks at the element's visible centre (clipped by its ancestors) whenever it has geometry and the toolkit's hit test does not report another element covering that point (an overlay, a sticky header); AT-SPI DoAction is used without geometry or when covered. When the toolkit's hit test stops at a container or answers with a sibling (GTK's scrolled window answers nothing for a tree view's rows, and GTK tree views leave the header row out of their coordinates), an element of a table is resolved by the table's own row geometry through the AT-SPI Table interface, and only another row at that point counts as a cover; anything else does not stop the click. A button or menu item that opens a modal dialog (gtk_dialog_run) would otherwise run the dialog's nested main loop inside the accessibility call, which blocks the app's AT-SPI connection. A menu item without geometry is refused, and a menu item's click must land on one of the app's popups (an item scrolled out of a long menu is refused rather than clicked in the window behind). While an earlier accessibility action call to an app has timed out and not returned, key and text input to it are refused with input_refused: GTK's key snooper makes a synchronous accessibility call per key event, and with a blocked connection the app would hang for good. A merely slow app is re-probed for up to 1 s and then gets the input with a warning.

• Chromium-based browsers need --force-renderer-accessibility. Web inputs expose Text but no EditableText, so setValue on them focuses the field with the pointer, checks that the click focused a writable field (else it fails without typing), selects all and types key events. Containers with more than 1000 children show only the children in view; scroll for more.

• Semantic click uses only an action named click/press/activate/toggle/jump/open/select (or selection/focus for selectable and editable elements); otherwise it fails and lists the available actions, which performSecondaryAction can run by name.

• Hover, right/middle and multi-clicks, held modifiers and press durations are part of the contract (CUA_ENGINE_EXTENSIONS is accepted for compatibility and changes nothing). Separate button press and release across calls (left_mouse_down/left_mouse_up) and native provider full-desktop coordinates are not supported; the adapters disable or reject them.

• The object mirror relies on apps announcing changes with AT-SPI events (names, states, children); a change without an event is seen at the latest after the 5 s rebuild. Text, value and geometry are always read live, per node. Large trees are bounded/truncated with warnings. App mapping uses PID plus frame title, size and position (a frame that encloses the window within decoration-sized margins) and rejects ties.

• Only mapped windows are listed. Scroll pixels approximate wheel notches. Screen text fences label untrusted data but do not solve prompt injection.

Development

scripts/gate runs everything a change must pass (formatting, lint, Rust, Node and Python tests, syntax checks and a secret scan) without a display, credentials or your home directory. It needs Rust with rustfmt and clippy, Node ≥22.13 and Python 3 with websockets. Focused tests:

cargo test --locked x11::tests::display_zero_denied
npm test --prefix surface

Desktop and installer checks that do not connect to a display:

python3 -m unittest discover -s tests -p test_install_desktop.py -v
python3 -m unittest discover -s desktop/tests -p test_ownership.py -v
python3 -m unittest discover -s desktop/tests -p test_native.py -v
python3 -m unittest discover -s desktop/tests -p test_browser.py -v

python3 -m unittest discover -s desktop/tests -p test_fast.py -v additionally creates and closes its own private fixture desktop. The legacy full desktop suite includes owner-display queries and panel tests; use the targeted checks above on a shared workstation.

On a private display

With the private fixture running and the same environment inherited:

node bench/integration.mjs
python3 bench/run.py --command "$CUA_ENGINE" --iterations 20 \
  --output bench/results/local.json --screenshot bench/results/fixture.png

The integration suite exercises the real MCP server, persistent App API, elicitation, Unicode insertion/paste, semantic and coordinate actions, unchanged real pointer/focus, repeated shared-memory captures, turn reset, XI2 abort, and both provider screenshot adapters. surface/test/real-apps.test.js is an opt-in regression suite against real apps (xed, gnome-calculator with LC_NUMERIC=de_DE.UTF-8, a launched Drawing) for coordinate refusal after a dialog opens, the dialog-close wait, single-line newlines, key-typing into dialogs, decimal-comma hints, shift-click range selection, hover and relative moves, the one-request tree-plus-screenshot, the clipboard in both directions and launchApp: cd surface && CUA_REAL_ENGINE=/path/to/linux-cua node --test test/real-apps.test.js on a private display with accessibility. surface/test/drawing.test.js is an additional opt-in regression for complete continuous strokes and refusal of an out-of-window path before drawing; run it with the same private-display environment. For occlusion coverage launch bench/fixture.py --guard on the same private display, then run with CUA_TEST_GUARD=1. CUA_TEST_SCREENSHOT=/absolute/path.png saves the engine capture.

bench/run.py checks approval gates and each task's resulting field/status text, and records per-call time, settle/observation time, response bytes, and per-task wall time. It measures local engine operations, not model latency or actual text-token usage. It accepts another NDJSON-compatible command for comparison; it does not execute or bundle Codex. bench/compare.py candidate.json reference.json refuses incompatible suites, models, dimensions, task counts or failed runs. A reference report must contain actual timings and independently checked outcomes, not the spec's fixed settle constants.

bench/model.mjs provides bounded native-provider inference loops on the synthetic fixture. Set the relevant ANTHROPIC_API_KEY or OPENAI_API_KEY securely in the environment, then select an explicit provider/model. This makes billable API requests and transmits fixture screenshots; subscription CLI OAuth sessions are not treated as API keys. It logs returned token usage, model/tool time, response sizes and independently checked task outcomes, never credentials. No automatic API retries occur. A dry run validates local setup without network calls and is deliberately not marked successful performance evidence:

node bench/model.mjs --provider anthropic --model claude-opus-5-5 \
  --iterations 5 --output bench/results/claude.json --dry-run
node bench/model.mjs --provider openai --model gpt-6.1-sol \
  --iterations 5 --output bench/results/gpt.json --dry-run
# Remove --dry-run only when intending to run paid inference.

The standard benchmark is benchmarks/run.mjs with its defaults: GPT-6.1-sol at medium effort in Codex (codex exec) draws a house in GNOME Drawing (apt package drawing) on a fresh private desktop and saves it; the saved PNG is verified for a non-trivial drawing in at least two colours. Run node benchmarks/run.mjs --label <change> --iterations N (add --record for a screen recording), then node benchmarks/report.mjs for pass rate and median wall, tool-call and tool time per label. The earlier tasks (reimburse, xed-note) and Claude (--agent claude --model claude-opus-5-5) stay available for cross-checks. Results append to $BENCH_ROOT/results.jsonl (default ~/.cache/linux-cua-bench). Wall time includes model inference, which dominates; compare tool time and tool calls for engine changes, and several iterations, because single runs vary by seconds.

Recorded results are in docs/measurements.md.

Licence

MIT, see LICENSE.