get_ui
Retrieves the visible Android UI element list with stable refs, text, IDs, and bounds so agents can target taps or text entry; returns a screenshot if no UI tree is available.
Instructions
The visible UI as a list of elements: [ref] (stable across calls), text, desc, id, centre, size, flags. Target elements by ref (index), text or id. Falls back to a screenshot when no tree is available.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "get_ui"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| all | No | Include layout-only nodes | |
| serial | No | adb serial; optional when exactly one device is connected |