screen_act
Do one thing on the page or app, like a person would: the pointer travels to what it clicks, fields are clicked before typing, scrolling uses the wheel; then a fresh look (no screenshot unless asked). action: click | double_click | hover | type | press | scroll | navigate | back | wait | say | focus | unfocus. Prefer ref from the latest look over x/y. While recording, use action 'say' with text before each step to narrate it (it becomes a caption). To show something off (a result, a chart, a setting), use action 'focus' with its ref: the take's camera zooms in on it and holds until the next focus or 'unfocus' (nothing on the page changes); without it the camera follows your clicks and typing. Pass steps (a list of these same objects, max 40) to run several in a row. In an app, if the user moves the mouse you are stopped: ask them, then pass resume=true.
response_detail (summary|compact|full), fields (dotted paths), cursor (next page) — see the server instructions.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| key | No | ||
| ref | No | ||
| url | No | ||
| text | No | ||
| clear | No | ||
| steps | No | ||
| action | No | ||
| amount | No | ||
| cursor | No | ||
| fields | No | ||
| resume | No | ||
| submit | No | ||
| seconds | No | ||
| direction | No | ||
| screenshot | No | ||
| response_detail | No |