ui_tree
List visible UI widgets with IDs, roles, names, and center coordinates to locate GUI controls for automation. Filter by app or window, or restrict to the active window.
Instructions
List visible widgets as [id] role "name" @(x,y) with center coordinates
in screenshot space. Filter by app/window name substring, or
active_window_only=True. Ids are valid until the next ui_tree call.
Apps only appear if accessibility is enabled (Chrome/Electron need
--force-renderer-accessibility).
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "ui_tree"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | ||
| window | No | ||
| max_depth | No | ||
| only_interactive | No | ||
| active_window_only | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| result | Yes |