Semantic desktop snapshot: monitors, workspaces, windows (address, class, title, at + size in global coords), active window, cursor, and layers: launchers (wofi/rofi), bars, notification popups, and on-screen keyboards are NOT windows and appear only there, with a best-effort kind and global geometry you can screenshot by region or click into. A listed layer is one the compositor tracks, not one you can see: it may be transparent or dormant, so screenshot when visibility matters. Call first; act on the addresses it returns.

Capture the focused monitor, a window (window: "active" or an address from desktop, cheapest for reading one app), or a region "x,y,WxH". Returns the image (or a file path to read) + JSON metadata with geometry/scale for pixel→global mapping. scale 0.1-1.0: optional deliberate downscale, usually leave unset. stable=true waits (up to 2s) until two consecutive frames match, so a capture right after an action is not taken mid-animation; metadata gains stable. Captures are fast JPEG by default; lossless=true returns PNG for pixel-exact work. Before clicking a small control, follow with zoom at the estimated point.

Native-resolution re-capture around a point: the precision step of the coarse-to-fine loop. Screenshot first, estimate the target's global x,y, zoom there, re-estimate on the zoomed image (scale ~1.0, so global = geometry[:2] + image_pixel), then click. size "WxH" in logical pixels (default 480x360) is clamped to the screen; window (an address from desktop) clamps to that window instead. The metadata echoes the requested point back as point; stable=true waits for the frame to settle first; lossless=true returns PNG.

Read a window's accessibility tree (AT-SPI) and return its elements with GLOBAL click points, so you can target a control by NAME with no screenshot and no pixel guessing. window is an address from desktop (default: the focused window). name filters to elements whose accessible name contains it (case-insensitive); actionable (default) keeps only interactive roles (buttons, entries, menu items, ...). Returns [{role, name, x, y, clickable}] where x,y is the click point: focus the window, then click it with pointer (the window must be visible to receive the click), or do both in one call with click_ui. Controls that carry a CURRENT VALUE also report it: value (text typed into an entry, or a slider/spinner number), percent for a slider's position, checked for a box or toggle. Password fields never report contents, and many dropdowns expose no value at all, so read the screen with screenshot when a rendered value matters. Not every app exposes a tree (terminals, and Electron/Chrome without --force-renderer-accessibility, expose little or nothing); when it does not, fall back to screenshot + zoom.

Set-of-Marks capture: a screenshot of the window WITH its accessible controls drawn as numbered red marks, plus a JSON legend mapping each number to the control's role, name, current value, and exact global click point. One glance replaces the estimate-zoom-estimate loop for every control the accessibility tree knows: read the number off the image and call click_ui(mark=N) (or pointer at the legend's x,y). window is an address from desktop (default: focused); name filters the marked controls. Falls back to the plain legend when ImageMagick is not installed, and to a fall-back-to-vision note when the app exposes no tree (then use screenshot + zoom).

The user's own Hyprland keybinds: combo, action, arg, and a description when the config provides one. This is how the desktop's owner drives it: to perform one of these workflows, call use_bind with the combo (it runs the bound action). An action of lua means the bind is a closure in a Lua Hyprland config, which nothing can run from outside: read its description and do the same thing with hypr or launch. NOTE: the keyboard tool canNOT trigger these compositor binds (synthetic keys reach apps, not Hyprland's bind matcher), so do not try to press them.

Block until a desktop event happens (real compositor events, not polling). event: 'window_open' | 'window_close' | 'workspace' | 'title_change' | 'layer_open' | 'layer_close' (layer-shell surfaces: launchers, notification popups; match on the namespace, e.g. 'wofi') | 'urgent' (a window demands attention) | 'screencast' (screen sharing started/stopped). match: optional case-insensitive substring filter over the event's fields (class/title/workspace name/address/namespace). timeout_s 1-60, default 10. Returns the event payload, or a timeout note; a filtered wait whose condition ALREADY holds (the window is already gone, the workspace already active, the layer already mapped) returns instantly with already: true instead of missing an event that fired before it could subscribe. Use it after actions with delayed effects: app startups, page loads that change a window title, a launcher bind that pops a layer.

Mouse. action='move' (x,y) | 'click' (optional x,y first; button left/right/middle; double=true) | 'drag' (x,y → to_x,to_y holding button) | 'scroll' (scroll_dy notches, positive = content down; optional x,y first).

Coordinates are GLOBAL logical pixels by default. Pass window (address, class/title substring, or a description like "the file browser") plus x_pct/y_pct in 0.0–1.0 to click RELATIVE to that window instead: (0.5,0.5) is its centre, (0.1,0.05) near its top-left. The window is focused first. This is the robust form — it survives the window moving or resizing.

then appends the result to this call so you skip a round-trip: 'desktop' a fresh snapshot, 'screenshot' a stable capture, 'ui' the focused window's elements with current values, 'none' (default) nothing.

Keyboard to the focused app. action='type' (text, unicode-safe) | 'key' (keys combo: 'ctrl+shift+t', 'esc', 'F5'; aliases enter/esc/tab/backspace/pgup/pgdn/arrows, else XKB keysyms). Pass window (an address from desktop) to focus that window first, so keystrokes land in the intended app rather than whatever currently holds focus. This drives shortcuts the focused application handles (ctrl+t, ctrl+l). It does NOT trigger Hyprland's own keybinds (super+...): those go through use_bind, and workspace/window actions through hypr. then ('desktop'|'screenshot'|'ui'|'none') appends the result to this call.

Click a control by its accessible NAME, or by a mark number from the last marks capture, in ONE call: the exact coordinate comes from the accessibility tree, the window is focused first, and the click goes through the real pointer (visible cursor), so no screenshot and no pixel estimation is spent. Pass exactly one of name (matched against window's controls, exact accessible name preferred, substring otherwise) or mark. An ambiguous name returns the candidates instead of guessing: disambiguate with index (0-based into that list) or a more specific name. Falls back with a note when the app exposes no tree (use screenshot + zoom + pointer then). then ('desktop'|'screenshot'|'ui'|'none') appends the result; 'ui' shows the click's effect on the controls in the same call.

Window/workspace ops over IPC (instant, no vision). action='workspace' (workspace: number/name/'special:name') | 'focus_window' (target: address) | 'move_window' (target + workspace, silent) | 'close_window' (target) | 'fullscreen' (target?) | 'toggle_floating' (target?) | 'dpms_on' / 'dpms_off' (display power; screenshots cannot capture a dark display — desktop() reports it and screenshot errors with a pointer here). then ('desktop'|'screenshot'|'ui'|'none') appends the result to this call.

Run command via Hyprland exec. Optional workspace placement (silent, works even for single-instance apps like browsers, whose window gets moved after it appears) and wait_s (1-30, default 8; raise for slow apps). Returns the new window's address/class/title/workspace, or a timeout note.

Run one of the user's own Hyprland keybinds by its combo (from the binds tool), e.g. 'SUPER+F'. This executes the bound action directly (the only reliable way: synthetic keypresses do not trigger compositor binds). Use it to drive the owner's configured workflows: launchers, layout shortcuts, scratchpads. then ('desktop'|'screenshot'|'ui'|'none') appends the result to this call (handy after a launcher bind).

Run an ordered list of actions in ONE call, so a click/type/enter micro-sequence costs one round-trip instead of several. Each step is {"op": "pointer"|"keyboard"|"click_ui"|"hypr"|"wait_for", ...that tool's args}, e.g. [{"op":"pointer","action":"click","x":800,"y":60}, {"op":"keyboard","action":"type","text":"hello","window":"0x.."}, {"op":"keyboard","action":"key","keys":"enter"}]. Every step runs, in order, regardless of what the desktop does in between; a step that raises stops the run and reports which one. To type into a specific window reliably give that keyboard step a window= (address, substring or description; it focuses first). Opt-in stop_on_change=true aborts between steps if a window opens/closes/moves or the workspace switches unexpectedly — useful when a dialog might appear, but it also fires on changes you intended (Ctrl+T opening a tab), so it is off by default. then observes the final state ('desktop' default, 'screenshot', 'ui', 'none').