Skip to main content
Glama

Act in the window

act
Destructive

Executes up to 32 verified UI actions on a Windows app window in the background, stopping at the first failed or unconfirmed step.

Instructions

Steps on the window of the last look, in the background first (Win32 messages, UI Automation, DevTools for Chromium), the real mouse/keyboard only as a fallback under the shared foreground token. One step, or up to 32 each with its own expect; stops at the first step not verified. Per step: ✓ verified, = already done, ~ unknown (never replayed: look again), ✗ failed or refused. interaction: true keeps the window in front for the whole call (shared foreground token, at most 30 s, given back at the end) for a menu that closes when the foreground goes back (a context menu opened by a real right click, then its command): selectors only, every step with its expect; look again afterwards.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "steps": "<steps>"
    },
    "name": "act"
  }
}

Input Schema

NameRequiredDescriptionDefault
stepsYes
interactionNoHold the foreground for these steps only (at most 30 s), released when the call ends
observationNoObservation id returned by look (or by the previous act): the window and tree the steps aim at; refused once another look replaced it

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv3.7.4
    • addedInput schema / properties / interaction
      Added value: +{
      +  "description": "Hold the foreground for these steps only (at most 30 s), released when the call ends",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / observation
      Added value: +{
      +  "description": "Observation id returned by look (or by the previous act): the window and tree the steps aim at; refused once another look replaced it",
      +  "type": "string"
      +}
  2. Changed2 schema fields changedv3.1.7
    • changedInput schema / properties / steps / items / description
      Previous value: -"One action on the last observation. kind: click (selector, or point + screenshotId of a native capture; button left|right|middle; count 1|2), setValue (selector, value), typeText (text; focus: selector), key (key: \"ctrl+s\", \"enter\"), select (selector, option), menu (path: \"File > Save\"), secondary (selector, action), scroll (point, screenshotId, scrollX, scrollY), drag (from, to, screenshotId). Optional: expect (postcondition), timeoutMs (0-15000), refresh (text|image|both), fallback (false forbids real mouse/keyboard), confirm and accord for sensitive actions (see the refusal)."New value: +"One action on the last observation. kind: click (selector, or point + screenshotId of a native capture; button left|right|middle; count 1|2), setValue (selector, value), typeText (text; focus: selector), key (key: \"ctrl+s\", \"enter\"), select (selector, option), menu (path: \"File > Save\"), secondary (selector, action), scroll (point, screenshotId, scrollX, scrollY), drag (from, to, screenshotId), hover (selector, or point + screenshotId: one mouse move, no button; give expect or refresh \"image\" to read the tooltip or menu it reveals). Optional: expect (postcondition), timeoutMs (0-15000), refresh (text|image|both), fallback (false forbids real mouse/keyboard), confirm and accord for sensitive actions (see the refusal)."
    • changedInput schema / properties / steps / items / properties / kind / enum
      Previous value: -[
      -  "click",
      -  "setValue",
      -  "typeText",
      -  "key",
      -  "select",
      -  "menu",
      -  "secondary",
      -  "scroll",
      -  "drag"
      -]New value: +[
      +  "click",
      +  "setValue",
      +  "typeText",
      +  "key",
      +  "select",
      +  "menu",
      +  "secondary",
      +  "scroll",
      +  "drag",
      +  "hover"
      +]
  3. First observedv0.1.0

TDQS

A3.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations, it discloses the execution strategy (background Win32/UIA/DevTools first, real input only as fallback), stopping behavior, per-step status meanings, and the shared foreground token limit. This is rich behavioral detail for a destructive, non-idempotent tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the main execution model and then adds the important interaction exception. It is dense and slightly run-on, but each clause carries useful operational information rather than filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex UI-automation tool with no output schema, it covers execution fallback, stopping behavior, status outcomes, and the special foreground-hold case. Some details around refusal handling and full parameter behavior remain in the schema rather than the description, but the core context is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67%, and the schema itself richly describes step kinds, controls, and expect. The description adds meaning around 'expect', step limits, and interaction constraints, but it does not fully explain all parameter semantics or compensate completely for the coverage gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the resource (the window of the last look) and the core operation (steps/actions), and distinguishes it from look by saying to look again afterwards. The verb 'steps' is a bit jargon-like, but the meaning is still clear in context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives some implied usage context: use it on the last observation, and look again afterwards. It also gives a specific condition for interaction: true for menus that close when foreground changes, but it does not explicitly compare act against siblings like window or look.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools