interact_with_ui
Perform UI interactions (tap, swipe, text input) on Android and iOS apps. Target elements by ID/text or coordinates. Automate mobile app testing with precise UI actions.
Instructions
Perform UI interactions like tap, swipe, or text input. Can target elements by ID/text or by coordinates.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| platform | Yes | Target platform | |
| action | Yes | Type of interaction to perform | |
| element | No | Element ID, resource ID, or text to interact with | |
| x | No | X coordinate for coordinate-based interaction | |
| y | No | Y coordinate for coordinate-based interaction | |
| text | No | Text to input (for input_text action) | |
| direction | No | Swipe direction (for swipe action) | |
| durationMs | No | Duration in milliseconds (for long_press and swipe, default: 300) | |
| deviceId | No | Device ID or name (optional) |