ui
Read a window's accessibility tree (AT-SPI) and return named controls with global click points, so agents can target elements without screenshots or pixel guessing.
Instructions
Read a window's accessibility tree (AT-SPI) and return its elements
with GLOBAL click points, so you can target a control by NAME with no
screenshot and no pixel guessing. window is an address from desktop
(default: the focused window). name filters to elements whose
accessible name contains it (case-insensitive); actionable (default)
keeps only interactive roles (buttons, entries, menu items, ...); with
a name that matches none of those, it keeps named elements in other
roles that have an accessibility action (a web app's list row), with
the action's name as action.
Returns [{role, name, x, y, clickable}] where x,y is the click point:
focus the window, then click it with pointer (the window must be
visible to receive the click, and focusing can move it, so read again
after focusing), or do both in one call with click_ui.
Controls that carry a CURRENT VALUE also
report it: value (text typed into an entry, or a slider/spinner
number), percent for a slider's position, checked for a box or
toggle. Password fields never report contents, and many dropdowns
expose no value at all, so read the screen with screenshot when a
rendered value matters. Not every app exposes a tree (terminals, and
Electron/Chrome without --force-renderer-accessibility, expose little
or nothing); when it does not, fall back to screenshot + zoom.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| window | No | ||
| actionable | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |