Describe a window's UI as elements {name, role, rect}. Target = window= (title substring; becomes the session anchor) or the current anchor; with neither, the human's foreground window and the response says so (target.source). Never focuses or raises anything. mode=full (default whenever a window is targeted: scoped walk, works on covered windows and hidden-desktop apps), mode=fast (foreground-only quick path, used only when nothing is targeted), mode=text (READ the visible text - the cheapest way to read content), mode=delta (changed elements since since_seq), mode=marks (DEBUG Set-of-Marks image), mode=selection (selected text of a name/role element). Elements capped at 150 (limit); text at 20000 chars.

Locate an element (name|role|description|text) and return center (always), rect (has_rect=true for UIA/cache hits), source, confidence, name, escalated (true = a network VLM call was paid). Target window = window= (anchored) or the session anchor; else the foreground. Under the default background policy nothing is focused or raised: the lookup is scoped to the window's own UIA subtree, so it works while the window is covered or lives on a hidden desktop.

Find an element and act on it in one call. Target window = window= (anchored) or the session anchor. Under the default background policy the window is NEVER raised and the cursor never moves: click = UIA Invoke or a posted click, type = ValuePattern / WM_SETTEXT with read-back, and the response reports {verified, focus_preserved, cursor_preserved}. Works on covered windows, Chromium/Electron pages and hidden-desktop apps. If a windowless control's own provider activates its window anyway (Chromium does on SetValue), the foreground is handed straight back to the window the human had and the response carries focus_guard. Identify the element by name|role (description needs vision and a screen-facing window). verified=false = dispatched but nothing visibly changed: ghost_see before retrying. background=true is accepted for compatibility; it is already the default behaviour.

Send a key to a window WITHOUT focusing it. Target = window= (anchored) or the session anchor. Under the default background policy the key is POSTED to the window's focused control: single keys (Enter, Tab, F5, arrows, any character) and the editing shortcuts Ctrl+C/X/V/A/Z (sent as semantic messages). Other modifier combos have no background path and error naming the alternative. For text prefer ghost_act action=type (read-back verified). Hold/release keys='down:Shift' / 'up:Shift' exist only under a foreground policy.

Structured, agent-planning view of a window's UI: every element with a stable id, name, role, rect, center, enabled flag, actionable flag, and the actions it accepts (click/type). actionable = interactable role AND currently enabled - so a greyed-out button reads actionable:false. Far cheaper than a screenshot to reason over; plan here, then ghost_act by name/role. Params: window? (title substring; omitted = foreground), actionable_only? (only enabled interactable elements), limit?.

Scroll. direction: up/down/left/right, amount = notches (default 3). Target = window= (anchored) or the session anchor: under the default background policy the wheel is POSTED to that window (pointer never moves, window never raised; works on covered and hidden-desktop windows). 'Until' mode: until_name/until_role scrolls repeatedly until that element is present, up to max_scrolls; returns found. Coord mode (x/y at the real pointer) needs a foreground policy.

Click-hold at a start point, move to an end point, release. Each endpoint may be raw coords (from_x/from_y, to_x/to_y) OR an element (from_name/from_role, to_name/to_role) resolved to its center - e.g. drag a list row onto another, or a slider handle.

Unified wait - use it instead of sleeping. for=element: an element (name/role) appears or disappears in the target window; returns the moment it does. for=value: an element's VALUE equals/contains/changes (forms, async fields, 'wait until the total updates'). for=navigate: load url in a browser window WITHOUT raising it - sets the address bar over UIA, presses Enter, returns when the window title changes (response: title_before, title_after, title_changed, ms); only a foreground focus policy takes the old focus-and-type path. for=idle: screen stable. for=text: click a target then wait for text. for=event: next foreground change. for=cond: JSONLogic poll. for=ms: a plain sleep - last resort, it costs its full length every run. Target window for element|value|idle|text|navigate = window= or the session anchor.

Extract structured data from the screen. Strategy: UIA name-matching first, then a single batched VLM call for any fields still unmatched. Returns extracted object, unmatched list, and vlm_attempted flag.

Assert a predicate about screen state. Fails (error) if not satisfied. text-present/text-absent: OCR text check. element-exists: element found. value-equals/value-contains: the element's actual value (ValuePattern) equals/contains 'text' - the fill-then-verify check.

Execute a declarative step-by-step flow in one round-trip. Each step: {op, ...params}. Op is any lean verb or legacy tool name. Retries each step on failure (max_retries). CHAINING: a param value of "${steps.N.path}" is replaced with a field from step N's result before dispatch - e.g. {op:'find',name:'Save'} then {op:'ghost_click_at', x:'${steps.0.center.x}', y:'${steps.0.center.y}'}. A whole-string ref keeps its type (number stays number). SPEED: name the window once; later steps can omit window= because the anchor follows that window by handle through title changes (a stale title still resolves and the response carries title_drift). Wait with {op:'ghost_wait', for:'element'|'value'|'navigate'} rather than for=ms - a fixed sleep costs its full length on every run.

Capture a screenshot of the window the agent is working in. Target = window= or the session anchor: that window is captured BY HANDLE (works while covered, on a hidden desktop, or via CDP for a routed browser) at max 768px JPEG q=75 (~20-100KB). Only with no anchor does it fall back to the foreground window. Pass name/role to crop to ONE element inside the target, or rect=[l,t,r,b] (great for VLM-in-the-loop checks). full=true: the whole screen at max 1280px JPEG (max_dim=0 = native-res lossless PNG). Always includes size_bytes.

Window management across the user's desktop AND Ghost's hidden desktops. op=list: every window (name, pid, hwnd, focused, state, surface=user|hidden) plus the current anchor. op=focus: under the default background policy this ANCHORS the window and does NOT raise it (the anchored verbs drive it without focus); only an operator-unlocked prefer_background/foreground policy raises it. op=anchor: name= sets the anchor, clear=true clears it, no args reports it. op=state: maximize|minimize|restore|close (name+state); under the background policy these never activate the window, so the user's focus stays put (restore is how a minimized window becomes actionable). op=launch: start exe - under the background policy the app starts on a hidden desktop, never on your screen, and is anchored (see target.surface in the response).

Run shell commands and drive persistent shells. THE way to run builds, git, CLIs, scripts, or open apps from a command line, and to edit files on machines with no file tools. op=run (default): one-shot - spawn shell, run 'cmd', return {output, exit_code, timed_out}. shell=bash (default) | sh | zsh | pwsh. op=open: start a persistent bash whose variables/cwd/env survive across commands (returns an 'id'). op=send: run 'cmd' in session 'id' (state persists). op=read: drain the rest of a command that hit its timeout (busy=true). op=list: show sessions. op=kill: end a session. Output is merged stdout+stderr, tail-capped at 24000 chars. Emergency-stop (ghost_stop) kills a runaway command. To START A NEW CLAUDE CODE SESSION: op=run cmd='gnome-terminal -- claude' (or your terminal of choice), then drive the terminal window with ghost_see / ghost_act / ghost_key. Disabled entirely when GHOST_SHELL=off.

Clipboard access. op=get (default): read text. op=set: write text.

Grounding + cache telemetry for debugging flow speed/reliability: which tier is winning (cache/UIA/OCR/VLM), VLM escalation rate, cache hit/miss counts, UIA-mirror stats. Call when a flow is slow or a find is unreliable.

Resume automation after ghost_stop. Clears the stop flag.

Emergency stop: halt all automation and release held modifier keys.

HTTP GET. Returns {status, body}.

HTTP POST. Returns {status, body}.

Launch an isolated browser for background automation. Each id gets its own process, profile, and DevTools port, so concurrent ghost processes never collide. browser= chrome | comet | edge | brave (default: first installed). mode=headless (invisible, default) | windowed (a real browser window, started on a hidden desktop - never on your screen, never takes focus; use it only for sites that behave differently headless). Never launched on your own desktop: a new Chromium window takes the foreground on creation. Tabs are driven individually via ghost_tab_* without focusing the window or moving the cursor.

Attach to a browser already running with --remote-debugging-port= (use for the user's own logged-in browser). Ghost never closes a browser it did not start, and drives background tabs without switching the user's active tab.

Close a browser ghost launched (attached browsers are only disconnected).

List open tabs with target ids, titles, URLs.

List Chromium-family browsers installed on this machine (chrome, comet, edge, brave).

Open a new background tab (never brings the tab or window to the front). Returns its id for the other ghost_tab_* tools.

Navigate a tab and wait for load. Runs in a background tab; the user's view never changes.

Click an element by CSS selector via a trusted synthetic mouse event inside the tab's renderer. ~1-2ms; no cursor movement; works in a tab that is not in front.

Focus an element by CSS selector and type into it. Layout-independent; does not touch the user's keyboard.

Send a key (Enter, Tab, ArrowDown, a, ...) to the tab's focused element. modifiers: Alt/Ctrl/Meta/Shift.

Visible text of an element, or the whole page when selector is omitted. Prefer over screenshots for reading content.

Evaluate a JavaScript expression in the tab and return its value (awaits promises).

PNG of a tab, rendered by that tab regardless of whether it is in front or focused.

Structured list of visible interactive elements with selectors and coordinates. Prefer this over screenshots for deciding what to click.

Scroll a tab or a scrollable element inside it.

Choose an option in a , firing the input/change events a real choice fires.

Wait until a CSS selector matches an element in the tab.

Close a tab by id.

Find a tab whose URL or title contains a substring.