Skip to main content
Glama
Sign In
Handy

Handy

by miisodev
OverviewSchemaSourceRelated ServersDiscussionsAdmin
JavaScript
Local

See Computer Screen (Mapped Coordinates + Window Zoom + Diff)

handy_see
Read-onlyIdempotent

Capture the screen or a window with mapped coordinates, live mouse position, cursor color, visual diffs, and numbered UI elements to locate and click controls precisely.

Instructions

Capture the computer screen (or a single desktop window via window) in real-time with mapped (x, y) coordinate rulers, dual-contrast grid labels, live mouse reticle, hex color under cursor, active window info, and visual diff since the previous capture (changedPercent, changedBounds). Use region or window to zoom into dense UI areas at 1:1 resolution while preserving global screen coordinates, and set highlightDiff: true to outline changed pixels. Set marks: true to run Set-of-Mark UI element detection: Handy outlines and numbers every detected button, field, icon, text line, and panel, and returns their exact bounds and click centres in elements — then act on one with { "type": "click", "mark": 3 } instead of estimating coordinates.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "handy_see"
  }
}

Input Schema

NameRequiredDescriptionDefault
gridNoOverlay mapped coordinate rulers and grid (default: true)
marksNoRun Set-of-Mark UI element detection: outline and number detected controls/fields/icons/text/panels on the image and return their exact bounds and click centres in `elements` (default: false)
regionNoOptional screen rectangle { x, y, width, height } to zoom into
windowNoOptional window title or app substring to capture at 1:1 resolution while keeping global coordinates
displayNoMonitor index (default: 0 primary)
gridStepNoCustom major grid spacing in screen pixels
maxMarksNoMaximum number of element marks to return when `marks` is true (default: 40)
maxWidthNoMax output image width in pixels (default: 1280); drawn coordinates remain exact screen pixels
markKindsNoRestrict marks to specific kinds: control, text, icon, region (default: all)
showMouseNoOverlay live mouse reticle and (x, y) badge (default: true)
minMarkScoreNoMinimum detection confidence 0..1 for a mark to be kept (default: 0.3)
highlightDiffNoDraw an emerald CHANGED bounding box around pixels that changed since the previous capture

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv1.3.0

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnly/idempotent/non-destructive; the description adds substantial behavior beyond that — zoomed captures still preserve global screen coordinates, downscaling via maxWidth keeps drawn coordinates exact screen pixels, and returns include `elements` with bounds and click centres plus `changedPercent`/`changedBounds`. One subtle caveat it does not call out is that diff output depends on a mutable 'previous capture' baseline, so consecutive identical calls can differ in content, though this is not a contradiction of the side-effect annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded and the paragraph reads in a sensible order (capture → zoom → diff → marks → act). It is long for one block, and the marks sentence largely duplicates the `marks` schema description, so a small amount of text does not earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter tool with no output schema, the description covers the important ground: return payload contents (`elements`, `changedPercent`, `changedBounds`), coordinate semantics under downscaling, and the follow-up action pattern. Given the annotation coverage and the fully described schema, an agent has enough to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real meaning about how `region` and `window` interact (1:1 zoom while keeping global coordinates) and what `marks` produces in `elements`, which the schema only restates tersely. The remaining parameters (display, gridStep, maxMarks, markKinds, minMarkScore, showMouse) are left entirely to the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a concrete verb and resource ('Capture the computer screen ... or a single desktop window') and enumerates what the capture carries (coordinate rulers, mouse reticle, hex color, window info, diff). It also separates this tool from the acting sibling by stating that marks let you 'act on one with { "type": "click", "mark": 3 } instead of estimating coordinates', making the see-vs-act split explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear conditional guidance for the main switches: use `region`/`window` to zoom into dense UI areas, set `highlightDiff: true` to outline changed pixels, set `marks: true` for element detection. What it lacks is any explicit when-not guidance or named alternatives (e.g. when to prefer the windows-listing or status sibling instead of capturing).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools