do_steps
Execute a known sequence of desktop UI actions in one call, waiting to inspect results until the end. The batch validates up front and aborts before starting if any step cannot finish.
Instructions
Run a short sequence of actions in ONE call and look once at the end. Every separate tool call costs a model round trip of several seconds while the action itself takes milliseconds, so a known sequence -- activate, click, type, press Return, see the result -- belongs here rather than in four calls. Steps run with their own look off; the picture is taken after the last one, or at the step that failed. Use single tools when the next action depends on what the last one revealed. The sequence is validated up front -- a call that cannot finish never starts.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| look | No | What to show you afterwards. Default "auto": wait for the screen to stop changing, measure how much this action changed, and attach a picture of the affected window only if something did change -- so a click that hit nothing costs no tokens and says so. "window" always attaches it, "screen" uses the whole desktop (slower, 6x the tokens), "region" uses look_at, false skips all of it. Use false for the middle of a sequence you are going to check at the end anyway. | auto |
| steps | Yes | Ordered actions. Each is {do: verb, ...the same arguments that tool takes}. Verbs: activate(target), click(x,y), move(x,y), drag(from_x,from_y,to_x,to_y), scroll(x,y,dy), type(target,text), key(target,combo), press(path,expect_name), set_text(path,text), wait_for(condition,target,timeout) -- same arguments as the wait_for tool; 'wait' is an alias -- and sleep(ms, 1-60000; prefer a wait_for condition over a guessed duration). A malformed step anywhere refuses the whole call naming that step, and nothing executes. Any step may carry retry: {"attempts": N, "on": [codes]} -- it reruns on those error codes (default: the world-moved set) and reports how many runs it took. | |
| look_at | No | Rectangle for look:"region", in screen pixels. Object form {x, y, width, height} or array form [x, y, width, height]. | |
| settle_max_s | No | How long to wait for the screen to stop changing before looking. Raise it for an app that animates slowly; set it to 0 to capture immediately. | |
| stop_on_error | No | Stop at the first failing step. Leave this true unless the steps are genuinely independent. |