desktop_actions
Execute up to 50 desktop actions—clicks, typing, scrolling, UI Automation—in one round trip, with immediate abort when the E-Stop triggers.
Instructions
Execute a sequence of up to 50 desktop actions (click, move, drag, scroll, type, paste, key, clipboard, focus, launch, wait, wait_for, screenshot, state, or UI Automation actions like invoke/set_value/get_text) in a single round trip. Aborts immediately at <1ms latency if user E-Stop triggers.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"steps": "<steps>"
},
"name": "desktop_actions"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| clip | No | ||
| scale | No | ||
| steps | Yes | Ordered action steps to execute. | |
| format | No | ||
| region | No | ||
| stable | No | ||
| window | No | ||
| display | No | ||
| quality | No | ||
| surface | No | ||
| evidence | No | ||
| settle_ms | No | ||
| deadline_ms | No | ||
| region_space | No | ||
| stability_ms | No | ||
| capture_after | No | ||
| require_window | No | ||
| expected_transitions | No | ||
| stop_on_window_change | No |