Semantic desktop snapshot: monitors, workspaces, windows (address, class, title, at + size in global coords), active window, cursor, and layers when layer-shell surfaces are up: launchers (wofi/rofi), bars, notification popups, and on-screen keyboards are NOT windows and appear only there, with a best-effort kind and global geometry you can screenshot by region or click into. Call first; act on the addresses it returns.

Capture the focused monitor, a window (window: "active" or an address from desktop, cheapest for reading one app), or a region "x,y,WxH". Returns the image (or a file path to read) + JSON metadata with geometry/scale for pixel→global mapping. scale 0.1-1.0: optional deliberate downscale, usually leave unset. stable=true waits (up to 2s) until two consecutive frames match, so a capture right after an action is not taken mid-animation; metadata gains stable. Captures are fast JPEG by default; lossless=true returns PNG for pixel-exact work. Before clicking a small control, follow with zoom at the estimated point.

Native-resolution re-capture around a point: the precision step of the coarse-to-fine loop. Screenshot first, estimate the target's global x,y, zoom there, re-estimate on the zoomed image (scale ~1.0, so global = geometry[:2] + image_pixel), then click. size "WxH" in logical pixels (default 480x360) is clamped to the screen; window (an address from desktop) clamps to that window instead. The metadata echoes the requested point back as point; stable=true waits for the frame to settle first; lossless=true returns PNG.

Read a window's accessibility tree (AT-SPI) and return its elements with GLOBAL click points, so you can target a control by NAME with no screenshot and no pixel guessing. window is an address from desktop (default: the focused window). name filters to elements whose accessible name contains it (case-insensitive); actionable (default) keeps only interactive roles (buttons, entries, menu items, ...). Returns [{role, name, x, y, clickable}] where x,y is the click point: focus the window, then click it with pointer (the window must be visible to receive the click), or do both in one call with click_ui. Controls that carry a CURRENT VALUE also report it: value (text typed into an entry, or a slider/spinner number), percent for a slider's position, checked for a box or toggle. Password fields never report contents, and many dropdowns expose no value at all, so read the screen with screenshot when a rendered value matters. Not every app exposes a tree (terminals, and Electron/Chrome without --force-renderer-accessibility, expose little or nothing); when it does not, fall back to screenshot + zoom.

Set-of-Marks capture: a screenshot of the window WITH its accessible controls drawn as numbered red marks, plus a JSON legend mapping each number to the control's role, name, current value, and exact global click point. One glance replaces the estimate-zoom-estimate loop for every control the accessibility tree knows: read the number off the image and call click_ui(mark=N) (or pointer at the legend's x,y). window is an address from desktop (default: focused); name filters the marked controls. Falls back to the plain legend when ImageMagick is not installed, and to a fall-back-to-vision note when the app exposes no tree (then use screenshot + zoom).

The user's own Hyprland keybinds: combo, action, arg, and a description when the config provides one. This is how the desktop's owner drives it: to perform one of these workflows, call use_bind with the combo (it runs the bound action). NOTE: the keyboard tool canNOT trigger these compositor binds (synthetic keys reach apps, not Hyprland's bind matcher), so do not try to press them.

Block until a desktop event happens (real compositor events, not polling). event: 'window_open' | 'window_close' | 'workspace' | 'title_change' | 'layer_open' | 'layer_close' (layer-shell surfaces: launchers, notification popups; match on the namespace, e.g. 'wofi') | 'urgent' (a window demands attention) | 'screencast' (screen sharing started/stopped). match: optional case-insensitive substring filter over the event's fields (class/title/workspace name/address/namespace). timeout_s 1-60, default 10. Returns the event payload, or a timeout note; a filtered wait whose condition ALREADY holds (the window is already gone, the workspace already active, the layer already mapped) returns instantly with already: true instead of missing an event that fired before it could subscribe. Use it after actions with delayed effects: app startups, page loads that change a window title, a launcher bind that pops a layer.

Mouse in global coordinates. action='move' (x,y) | 'click' (optional x,y first; button left/right/middle; double=true) | 'drag' (x,y → to_x,to_y holding button) | 'scroll' (scroll_dy notches, positive = content down; optional x,y first). then appends the result to this call so you skip a round-trip: 'desktop' a fresh snapshot, 'screenshot' a stable capture, 'ui' the focused window's elements with current values, 'none' (default) nothing. allow_auth=true overrides the refusal to click over a system authentication dialog.

Keyboard to the focused app. action='type' (text, unicode-safe) | 'key' (keys combo: 'ctrl+shift+t', 'esc', 'F5'; aliases enter/esc/tab/backspace/pgup/pgdn/arrows, else XKB keysyms). Pass window (an address from desktop) to focus that window first, so keystrokes land in the intended app rather than whatever currently holds focus. This drives shortcuts the focused application handles (ctrl+t, ctrl+l). It does NOT trigger Hyprland's own keybinds (super+...): those go through use_bind, and workspace/window actions through hypr. then ('desktop'|'screenshot'|'ui'|'none') appends the result to this call. allow_auth=true overrides the default refusal to type into a password field or a system authentication dialog (only when a human intends that credential entry).

Click a control by its accessible NAME, or by a mark number from the last marks capture, in ONE call: the exact coordinate comes from the accessibility tree, the window is focused first, and the click goes through the real pointer (visible cursor, same panic-kill guarantees), so no screenshot and no pixel estimation is spent. Pass exactly one of name (matched against window's controls, exact accessible name preferred, substring otherwise) or mark. An ambiguous name returns the candidates instead of guessing: disambiguate with index (0-based into that list) or a more specific name. Falls back with a note when the app exposes no tree (use screenshot + zoom + pointer then). then ('desktop'|'screenshot'|'ui'|'none') appends the result; 'ui' shows the click's effect on the controls in the same call. allow_auth=true overrides the refusal to click a system authentication dialog.

Window/workspace ops over IPC (instant, no vision). action='workspace' (workspace: number/name/'special:name') | 'focus_window' (target: address) | 'move_window' (target + workspace, silent) | 'close_window' (target) | 'fullscreen' (target?) | 'toggle_floating' (target?). then ('desktop'|'screenshot'|'ui'|'none') appends the result to this call.

Run command via Hyprland exec. Optional workspace placement (silent, works even for single-instance apps like browsers, whose window gets moved after it appears) and wait_s (1-30, default 8; raise for slow apps). Returns the new window's address/class/title/workspace, or a timeout note.

Run one of the user's own Hyprland keybinds by its combo (from the binds tool), e.g. 'SUPER+F'. This executes the bound action directly (the only reliable way: synthetic keypresses do not trigger compositor binds). Use it to drive the owner's configured workflows: launchers, layout shortcuts, scratchpads. then ('desktop'|'screenshot'|'ui'|'none') appends the result to this call (handy after a launcher bind). Refused while HYPRUSE_CONFINE is set: a bind runs an arbitrary compositor action that cannot be scoped to a window.

Run an ordered list of actions in ONE call, so a click/type/enter micro-sequence costs one round-trip instead of several. Each step is {"op": "pointer"|"keyboard"|"click_ui"|"hypr"|"wait_for", ...that tool's args}, e.g. [{"op":"pointer","action":"click","x":800,"y":60}, {"op":"keyboard","action":"type","text":"hello","window":"0x.."}, {"op":"keyboard","action":"key","keys":"enter"}]. With stop_on_change (default) the run stops, best-effort, when it notices a STRUCTURAL change between steps that the step did not intend: a window opening (e.g. a dialog), closing, or moving, a switch to an unexpected workspace, or a seat-taking layer surface (a launcher or on-screen keyboard) appearing, so later steps do not act on stale state. Notification popups and bars are not treated as changes. It does NOT catch a bare focus change, so to type into a specific window reliably give that keyboard step a window= address (it focuses first). Bounded to 20 steps and ~30s total. then observes the final state ('desktop' default, 'screenshot', 'ui', 'none').