Skip to main content
Glama
Sign In
Handy

Handy

by miisodev
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Local

Control Mouse & Keyboard (Single or Chained)

handy_act
Destructive

Executes native mouse, keyboard, clipboard-paste, window-focus, and visual-wait steps in one chained call, enabling agents to automate desktop UI actions reliably.

Instructions

Execute native mouse, keyboard, clipboard-paste, window-focus, and smart visual-wait controls in real-time. Supports chaining multiple steps sequentially in one call — e.g. [{ "type": "focus_window", "title": "Code" }, { "type": "click", "x": 420, "y": 260 }, { "type": "paste", "text": "Hello!" }, { "type": "wait_for_settled" }]. Steps support coordinateSpace: "screen" | "normalized" | "window", and any pointer step accepts mark: N to target a detected UI element from the last handy_see({ marks: true }) instead of raw x/y. Set screenshotAfter: true to receive an updated coordinate-mapped screenshot with numbered action-trail markers, or verifyChange: true for a cheap did-anything-happen diff.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "actions": "<actions>"
    },
    "name": "handy_act"
  }
}

Input Schema

NameRequiredDescriptionDefault
gridNo
marksNoRun Set-of-Mark element detection on post-chain/mid-chain screenshots and return `elements`
actionsYesOrdered array of steps (move, click, double_click, triple_click, right_click, drag, scroll, type, paste, key, focus_window, wait, wait_for_change, wait_for_settled, wait_for_color, screenshot)
displayNoTarget display index for normalized coordinates or screenshotAfter
failsafeNoAbort immediately if a human slams the cursor to the top-left (0,0) corner (default: true)
maxMarksNoMax marks when `marks` is true (default: 40)
maxWidthNo
settleMsNoWait in ms before taking screenshotAfter (default: 50)
showMouseNo
showTrailNoDraw numbered amber markers (#1, #2, drag lines) on post-action screenshots (default: true)
stepDelayMsNoSettling delay in ms between steps so native UI focus updates reliably (default: 15)
verifyChangeNoCapture before and after the chain and report `change`/`changed` so you learn whether anything visually moved without requesting an image (default: false)
highlightDiffNoOutline changed pixels since the previous capture on post-action screenshots (default: false)
screenshotAfterNoCapture and return an annotated screen image after the chain completes (default: false)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.3.0

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=true, idempotentHint=false, openWorldHint=true, so the risk profile is known without the description. The description adds genuinely useful behavioral context — sequential chaining, coordinate spaces, mark-based targeting, and a cheap `verifyChange` diff instead of an image — but never warns that these steps act on the live desktop, that focus changes affect the user's machine, or mentions the failsafe to (0,0) abort. Meaningful additions on top of annotations, with notable omissions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the capability statement, then one working example chain, then a compact second sentence covering coordinate spaces, mark targeting, and the two screenshot/verify flags. Dense but every clause carries information; nothing is redundant with the title or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 14-parameter, 16-action union with no output schema, the description covers the payload shape, the targeting model, and the result-affecting flags, which is what an agent needs to call it correctly. Residual gaps are the return envelope beyond the mentioned flags and the permissions prerequisite, neither of which is documented here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 79%, and the description still adds semantics the schema alone does not surface at a glance: that `mark: N` refers to elements from the last `handy_see({ marks: true })` rather than raw x/y, the three coordinate spaces, and what `screenshotAfter` and `verifyChange` actually return. It illustrates with a concrete four-step chain, which clarifies how the oneOf action variants compose in practice.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Names a specific verb family and resource set: "Execute native mouse, keyboard, clipboard-paste, window-focus, and smart visual-wait controls in real-time." It also carves out its distinct role versus handy_see by pointing at `handy_see({ marks: true })` as the source of marks, so the agent can separate acting from looking.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context for when to use the chained form (multiple steps sequentially in one call) and when to reach for the optional outputs (`screenshotAfter` vs `verifyChange` for a cheap did-anything-happen diff). It does not state preconditions such as needing accessibility/screen-recording permission (there is a sibling handy_permission) or when to prefer discrete calls over a chain, so it falls short of explicit when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools