State-Tool
Get the current screen state: interactive UI elements with label indices, types, names, tap coordinates, and device context. Use vision for annotated screenshot. Call first before interacting.
Instructions
Get current screen state: interactive UI elements with label indices, types, names, and tap coordinates. Returns device context (current app, keyboard visibility, screen size). Set use_vision=True for annotated screenshot. Always call this first before interacting.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "State-Tool"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| use_vision | No |