act
Run ordered click, type, key, and menu steps in Windows apps in the background, verify each result, and stop at the first error or unknown outcome.
Instructions
Act in the background: steps run in order (stops at the first error or unknown outcome), then the updated state (then: auto = list + image if a step is not verified; both, image, elements, changes, none). Per-step report: ✓ invoked or effect read, = already done (nothing sent), ~ unknown outcome (batch stopped: inspect the state), ✗ failed or not reached; then the summary. Steps (do): click, double_click, right_click, type (appends text; no target = focused element), set (replaces the text), clear, key (keys "ctrl+s", "enter", "alt+f4"), select (option in a list), check, uncheck, toggle, expand, collapse, scroll (direction, amount), focus, menu (path "File > Open"), wait (text appeared, or gone=true; timeout), drag (target → to), hover. expect: text expected after the step. check: wanted state of an element ({"target":"Name","value":"Alice"}; also contains, checked, selected, expanded, focused, enabled, exists): already true → step skipped, read back after → ✓, otherwise ✗ not reached. confirm=true: paying, sending, publishing, permanent deletion… (refused otherwise, before any input); only on the user's explicit order or "yes" for THIS action. Outward ones (send, pay, publish…) also need accord: the token from the refusal, after the user's own yes. target: [N] from the last look, visible name ("Save", "button:OK", "edit:Name") or "x,y" in image pixels. Example: steps=[{"do":"set","target":"Name","text":"Alice"},{"do":"click","target":"OK"}]. Routines: remember="name" saves the batch if it is verified (✓/=, targets by name; {{x}} = variable text, only in text and check.value/contains; typed text made only of {{x}}); routine="name" (without steps) replays it, params={"x": "…"}; remember="name" with steps=[] forgets it; look lists them. foreground=true: window kept in front for the whole batch, real mouse/keyboard when the background is impossible or ignored (drag, hover, WPF/UWP/Chromium shortcuts, Tk); only with the user's consent. Panels window="quick settings", "notifications", "start": foreground=true only.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| then | No | auto | |
| steps | No | ||
| params | No | ||
| window | No | ||
| routine | No | ||
| remember | No | ||
| settle_ms | No | ||
| foreground | No |