interact
Script UI interactions using semantic selectors (click, type, key, tab, hover, wheel, drag) and receive emitted intent messages and the updated accessibility tree. Optionally capture a screenshot preview.
Instructions
Drive a UI through scripted interactions (click, type, key, tab, hover, wheel, drag — by semantic selector, never coordinates). Returns the emitted intent messages and the access tree afterwards; set screenshot=true for a preview image.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| size | No | ||
| steps | Yes | An array of interaction steps, e.g. `[{"click":{"role":"button","name":"Add"}}]`. | |
| theme | No | ||
| screenshot | No | Attach a preview image of the UI after the steps. | |
| description | Yes | The UI description: a `fenestra/1` JSON object. |