screen_act
Do one thing on the page or app, like a person would: the pointer travels to what it clicks, fields are clicked before typing, scrolling uses the wheel; then a fresh look (no screenshot unless asked). action: click | double_click | hover | type | press | scroll | navigate | back | wait | say | focus | unfocus | highlight | hide | unhide. Prefer ref from the latest look over x/y. You edit the take as you record it: the cut is finished the moment you stop, so decide everything here. While recording, use action 'say' with text before each step to narrate it (it becomes a caption and a timed line of the saved script, ready to be voiced). To show something off (a result, a chart, a setting), use action 'focus' with its ref, several refs framed together, or an exact area x, y, w, h (it fills the view); pad adds room, zoom 1.1-4 says how close outright, seconds how long: the take's camera zooms in and holds that long, or until the next focus or 'unfocus' (nothing on the page changes); without it the camera follows your clicks and typing. The result's shots gives each focus's zoom and view (the part of the page in frame, CSS pixels): check it and focus again if it is not what a viewer should see. Before recording, a focus only reports its shot: rehearse your framing. To point out or copy words, use action 'highlight' with the ref and the text in it, and color, seconds, lift, zoom_in if you like: the words are selected by a drag (press Control+C, Meta+C on a Mac, to copy) and the take marks them, lifted off the page (in an app the element is marked, nothing is selected). To keep something private (an e-mail, a key, a customer's name), use action 'hide' with its ref (and the text in it to hide only those words), or an area x, y, w, h: it is blurred in the take from then, following it as the page scrolls, until 'unhide' (with the ref, or none for everything); hidden before recording, it is hidden from the first frame. While recording every result says at: the take's clock in seconds. Pass steps (a list of these same objects, max 40) to run several in a row. In an app, if the user moves the mouse you are stopped: ask them, then pass resume=true.
response_detail (summary|compact|full), fields (dotted paths), cursor (next page) — see the server instructions.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| h | No | ||
| w | No | ||
| x | No | ||
| y | No | ||
| key | No | ||
| pad | No | ||
| ref | No | ||
| url | No | ||
| why | No | ||
| lift | No | ||
| refs | No | ||
| text | No | ||
| zoom | No | ||
| clear | No | ||
| color | No | ||
| steps | No | ||
| action | No | ||
| amount | No | ||
| cursor | No | ||
| fields | No | ||
| resume | No | ||
| submit | No | ||
| seconds | No | ||
| zoom_in | No | ||
| direction | No | ||
| screenshot | No | ||
| response_detail | No |