act
Executes up to 8 mouse, keyboard, scroll, or wait commands on a fresh observation, returning the next desktop state or image for agent control.
Instructions
Execute up to 8 hardware input actions on a fresh observation. Return next state/image.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"observation_id": "<observation_id>",
"actions": "<actions>"
},
"name": "act"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| image | No | ||
| actions | Yes | ||
| observation_id | Yes |