State-Tool
Get the current screen state: interactive UI elements with label indices, types, names, tap coordinates, and device context. Use vision for annotated screenshot. Call first before interacting.
Instructions
Get current screen state: interactive UI elements with label indices, types, names, and tap coordinates. Returns device context (current app, keyboard visibility, screen size). Set use_vision=True for annotated screenshot. Always call this first before interacting.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| use_vision | No |