Skip to main content
Glama

screenstudio_desktop_action

Destructive

Automate macOS app interactions by clicking, typing, scrolling, dragging, or pressing keys on a window to record or test workflows.

Instructions

One visible native step on an observed window: focus, move, click, doubleClick, drag, taps (a burst of count taps every intervalMs at one point, for mash buttons), hold (press for ms, then release), type, key (with modifiers, e.g. {key:'k',modifiers:['command']}; keys follow the person's keyboard layout) or scroll (optional x/y to scroll there; otherwise the pointer stays if it is over the window or moves to its middle). Coordinates are window-local points including the title bar. Pointer glides at a human pace and settles before clicking. Input stops if the window loses focus, another window covers the point, or the person moves the mouse. Terminals, password managers and System Settings are off limits unless SCREENSTUDIO_ALLOW_APPS names them; system shortcuts (command+space, command+tab, command+q) are never sent.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
paceNo
actionYes
observationTokenYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.5.0

TDQS

A3.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare readOnlyHint=false, destructiveHint=true, openWorldHint=true, and the description adds substantial context beyond them: the human-pace pointer glide, the stop conditions (focus loss, occlusion, user mouse movement), the app allow-list restriction, and the hard exclusion of system shortcuts like command+space/tab/q. This is unusually rich behavioral disclosure for a mutating UI tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core purpose is front-loaded in the first clause, and each sentence carries real information, but the whole definition is delivered as one very long run-on paragraph mixing action semantics, safety limits, and pointer physics, which makes it hard to scan. Structuring the action list, the pointer behavior, and the restrictions into distinct sentences or bullets would preserve the content while improving retrieval.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, open-world UI automation tool with no output schema, the description covers safety, side effects, restrictions, and per-action semantics well. It stops short of saying how observationToken is obtained (presumably from screenstudio_desktop_inspect/screenshot) or what the call returns on success or on being interrupted, which would close the remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description has to carry the parameters, and it largely does: taps is explained as 'a burst of count taps every intervalMs at one point, for mash buttons,' hold as 'press for ms, then release,' key as taking modifiers with a layout caveat, and scroll as taking optional x/y with a described pointer fallback. The top-level pace and observationToken parameters are never explained, which is the only real gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific resource and scope — 'One visible native step on an observed window' — and enumerates the exact step types (focus, move, click, doubleClick, drag, taps, hold, type, key, scroll), so an agent knows precisely what single unit of work this performs. It does not, however, distinguish itself from the sibling screenstudio_desktop_perform, which sounds like an overlapping performer, nor from desktop_inspect/screenshot.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is implied through runtime conditions rather than tool selection: it explains that input stops if focus is lost, a point is covered, or the person moves the mouse, and that terminals/password managers/System Settings are off limits unless SCREENSTUDIO_ALLOW_APPS names them. There is no statement of when to prefer this over desktop_perform or when to observe first with desktop_inspect/screenshot.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.