ui_tree
Extracts a window's controls as compact, one-line text with refs so AI agents can inspect desktop UIs; filter lines or include screen rectangles.
Instructions
A window's controls as compact text, one line per control with a ref (e12). Long lists show 25 items; filter prints matching lines only (refs still cover the window); rects adds screen rectangles. If it shows almost nothing (games, Qt, canvas apps), use ui_ocr.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"title": "<title>"
},
"name": "ui_tree"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| rects | No | add @x,y wxh | |
| title | Yes | part of the window title (case-insensitive; an exact title wins) or a handle from list_windows (0x...) | |
| filter | No | only lines containing this text | |
| max_nodes | No | default 300 |