Act in the window
actExecutes up to 32 verified UI actions on a Windows app window in the background, stopping at the first failed or unconfirmed step.
Instructions
Steps on the window of the last look, in the background first (Win32 messages, UI Automation, DevTools for Chromium), the real mouse/keyboard only as a fallback under the shared foreground token. One step, or up to 32 each with its own expect; stops at the first step not verified. Per step: ✓ verified, = already done, ~ unknown (never replayed: look again), ✗ failed or refused. interaction: true keeps the window in front for the whole call (shared foreground token, at most 30 s, given back at the end) for a menu that closes when the foreground goes back (a context menu opened by a real right click, then its command): selectors only, every step with its expect; look again afterwards.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"steps": "<steps>"
},
"name": "act"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| interaction | No | Hold the foreground for these steps only (at most 30 s), released when the call ends | |
| observation | No | Observation id returned by look (or by the previous act): the window and tree the steps aim at; refused once another look replaced it |