Skip to main content
Glama
Sign In
Handy

Handy

by miisodev
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Local

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
HANDY_BACKENDNoSet to "virtual" to use the built-in 1280x800 multi-window interactive virtual desktop backend for headless CI and agent sandboxing.

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

Handy provides real-time computer screen vision with mapped (x, y) coordinates, window-scoped capture (window), active window metadata, pixel-under-cursor color, and frame-to-frame visual diffs (handy_see), plus real-time native mouse, keyboard, clipboard, and window control supporting chained multi-step actions and window-relative coordinates (handy_act, handy_windows, handy_clipboard). Always call handy_see before clicking unfamiliar UI elements, prefer paste for long or multi-line text, use wait_for_change / wait_for_settled / wait_for_color instead of blind sleeps, and chain sequential steps in a single handy_act call.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tools
{
  "listChanged": true
}
prompts
{
  "listChanged": true
}
resources
{
  "listChanged": true
}

Tools

Functions exposed to the LLM to take actions

NameDescription
handy_seeA

Capture the computer screen (or a single desktop window via window) in real-time with mapped (x, y) coordinate rulers, dual-contrast grid labels, live mouse reticle, hex color under cursor, active window info, and visual diff since the previous capture (changedPercent, changedBounds). Use region or window to zoom into dense UI areas at 1:1 resolution while preserving global screen coordinates, and set highlightDiff: true to outline changed pixels. Set marks: true to run Set-of-Mark UI element detection: Handy outlines and numbers every detected button, field, icon, text line, and panel, and returns their exact bounds and click centres in elements — then act on one with { "type": "click", "mark": 3 } instead of estimating coordinates.

handy_actA

Execute native mouse, keyboard, clipboard-paste, window-focus, and smart visual-wait controls in real-time. Supports chaining multiple steps sequentially in one call — e.g. [{ "type": "focus_window", "title": "Code" }, { "type": "click", "x": 420, "y": 260 }, { "type": "paste", "text": "Hello!" }, { "type": "wait_for_settled" }]. Steps support coordinateSpace: "screen" | "normalized" | "window", and any pointer step accepts mark: N to target a detected UI element from the last handy_see({ marks: true }) instead of raw x/y. Set screenshotAfter: true to receive an updated coordinate-mapped screenshot with numbered action-trail markers, or verifyChange: true for a cheap did-anything-happen diff.

handy_windowsA

List visible desktop windows with their titles, owning apps, screen bounds { x, y, width, height }, and focused state (action: 'list'), or bring a specific window to the foreground (action: 'focus').

handy_clipboardA

Read current plain-text clipboard contents (action: 'read', works in full and view_only modes) or write Unicode text directly to the system clipboard (action: 'write', requires full control mode).

handy_statusA

Check whether Handy permission is currently ON or OFF, which native OS backend is active, connected monitor dimensions, active window, and current mouse position (when enabled).

handy_permissionA

Inspect or turn OFF Handy computer control permissions. Turning Handy OFF (action: 'off') immediately disables screen capture and mouse/keyboard control and releases any held inputs. When disabled by the user, re-enabling requires running handy on (or /handy:on) unless HANDY_ALLOW_MCP_ENABLE=true.

Prompts

Interactive templates invoked by user choice

NameDescription
computer-controlGuide an LLM through accurate coordinate-mapped screen reading and chained mouse/keyboard control.

Resources

Contextual data attached and managed by the client

NameDescription
handy-statusLive JSON state of Handy permission toggle, OS backend, active window, and displays.
handy-windowsLive list of visible desktop windows with titles, apps, and bounds (requires Handy ON).
handy-current-screenReal-time PNG screenshot with mapped coordinates and live mouse position (requires Handy ON).
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.1/5.0

Scored across 6 tools

Disambiguation4/5

Each tool maps to a distinct capability (observe, act, enumerate windows, clipboard, status, permissions), and descriptions give clear guidance on when to use handy_see vs handy_act. There is minor overlap: window focusing exists both in handy_act as a step type and as handy_windows 'focus', and paste in handy_act overlaps with handy_clipboard write, while handy_status vs handy_permission both report permission state.

Naming Consistency4/5

All six tools share a consistent 'handy_' prefix with snake_case suffixes. The suffixes mix verbs (see, act) with nouns (windows, clipboard, permission, status), which is a small deviation but still predictable and readable as a toolset.

Tool Count5/5

Six tools is tightly scoped for a computer-control server: observe, act, windows, clipboard, status, and permissions. Each one earns its place and there is no redundant filler.

Completeness4/5

Coverage spans the core loop of screenshot -> act -> verify, plus window enumeration, clipboard I/O, and permission control. Gaps remain for window management (move/resize/close), multi-monitor enumeration, and richer clipboard formats, but agents can work around these.

Maintenance

ActivityMaintained
ResponsivenessNo recent community issues

Resources