act_window
Run a sequence of UI steps in one window, stopping at the first failure. Performs clicks, typing, keys, set, actions, scroll, drag, focus, and wait by element ref or screenshot pixels.
Instructions
Run steps in one window, in order, stopping at the first that fails: click, type, key, set, action, scroll, drag, focus, wait. Targets are e-refs from read_window or [x, y] pixels of its read_screenshot; drag takes pixels of a whole-screen read_screenshot. Element clicks, set and action never bring the window to the front; keys and typing don't on Hyprland, elsewhere they focus it. Mouse steps ([x, y] clicks, right or double clicks, scroll, drag) need a focus step first, which shows the window to the user. focused: true sends keys to the focused window, refused if the focus moved. then: tree (default after read_window) or screenshot (brings the window forward except on Hyprland) shows the result.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"window": "<window>",
"steps": "<steps>"
},
"name": "act_window"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| then | No | ||
| steps | Yes | ||
| window | Yes | Window id from read_windows |