ai_action
Execute UI actions on Android devices using natural language commands to automate testing and interaction workflows.
Instructions
Perform an action on the UI of the android box (natural language instruction).
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| boxId | Yes | ID of the box | |
| instruction | Yes | Direct instruction of the UI action to perform, e.g. 'click the login button' | |
| background | No | Contextual background for the action, to help the AI understand previous steps | |
| includeScreenshot | No | Whether to include screenshots in the action response (default false) | |
| outputFormat | No | Output format for screenshot URIs (default 'base64') | |
| screenshotDelay | No | Delay after performing the action before the final screenshot, e.g. '500ms' |