Skip to main content
Glama
sidebutton

computer-use

by sidebutton

plugin-computer-use

A persistent stdio MCP server that exposes the Anthropic computer-use action surface (screenshot, click, move, keyboard, clipboard, batch) against the SideButton agent desktop on DISPLAY=:10.

This repo is the scaffold + dispatch core for the Computer Use epic (SCRUM-1399). It is delivered by SCRUM-1397:

  • the long-lived stdio MCP server loop (initialize / tools/list / tools/call),

  • the ported computer.py dispatch base (DISPLAY targeting, screenshot → base64 PNG, coordinate scaling, single-owner lock, xdotool runner),

  • the full tool surface declared so tools/list returns it,

  • screenshot wired end-to-end as the proof action.

The individual tool bodies land in sibling tickets (SCRUM-1400…1405) and hosting this as a runtime: "service" plugin is SCRUM-1406.

Why a persistent server

The current SideButton plugin model (the-assistant packages/server/src/plugins) spawns a fresh, stateless handler process per tools/call and SIGKILLs it at a 30s timeout. That cannot host the computer-use surface, which needs cross-call state: a held mouse button (left_mouse_downleft_mouse_up), the screenshot→coordinate session, session grants, and holds up to ~100s. So this is a single, long-lived child process that speaks MCP over stdio.

Related MCP server: desktop-touch-mcp

Tool surface

24 tools, grouped by the sibling ticket that owns each body. The capture group (screenshot, zoom, SCRUM-1400), the click group (left_click / right_click / middle_click / double_click / triple_click, SCRUM-1401), the keyboard group (type / key / hold_key, SCRUM-1403), and the clipboard + session group (SCRUM-1404) are implemented; the rest are declared and return a clear pending-owner error until their ticket lands. Full input schemas: docs/computer-use-mcp-tools-schema.md.

Group

Ticket

Tools

capture

SCRUM-1400

screenshot ✅, zoom

click

SCRUM-1401

left_click ✅, right_click ✅, middle_click ✅, double_click ✅, triple_click

move / drag / scroll

SCRUM-1402

mouse_move, left_click_drag, scroll, left_mouse_down, left_mouse_up

keyboard

SCRUM-1403

type ✅, key ✅, hold_key

clipboard + session

SCRUM-1404

read_clipboard ✅, write_clipboard ✅, request_access ✅, list_granted_applications ✅, open_application ✅, switch_display

utility / batch

SCRUM-1405

computer_batch, wait, cursor_position

Clipboard + session behaviour (SCRUM-1404)

The macOS session/permission model has no XFCE/Xvfb equivalent, so these degrade gracefully instead of erroring — keeping cross-runner (macOS-authored) skills working — while honouring the native grant flags so call shapes match:

  • request_access auto-grants the requested apps (no compositor dialog), records the clipboardRead / clipboardWrite / systemKeyCombos flags (additive across calls), and returns screenshotFiltering: false.

  • list_granted_applications echoes the allowlist + active grant flags.

  • read_clipboard / write_clipboard shell out to xclip -selection clipboard, gated on the clipboardRead / clipboardWrite grants (a call without the grant returns an isError result, matching native).

  • open_application is best-effort window focus (wmctrl -a, then xdotool search --name … windowactivate); the primary target is the single RDP window. With neither binary installed it returns a non-error no-op note.

  • switch_display is a no-op on the single Xvfb :10 and reports the current display (accepts "auto").

Surface count. This is the 24-tool surface the epic (SCRUM-1399) specifies. The clipboard + session group follows the explicit enumeration in SCRUM-1404 (read_clipboard / write_clipboard split + list_granted_applications), which is the 2-tool delta over the work plan's interim count of 22. src/tools.py is the single source of truth; docs/computer-use-mcp-tools-schema.md (AC4) is generated from it.

Bare names + collisions. Names are the canonical Anthropic action ids. screenshot, type, scroll, wait, click collide with core SideButton MCP tools, and the current loader drops the entire plugin on any collision. That is fine standalone (this server owns its namespace); namespacing on aggregation is deferred to SCRUM-1406 (recommended: bare names in the child, prefix/slug-namespace on the host).

Layout

plugin-computer-use/
├── plugin.json        # generated service-plugin manifest (proposes runtime:"service")
├── src/
│   ├── server.py      # stdio MCP loop: initialize / tools/list / tools/call
│   ├── computer.py    # dispatch base (ported computer.py)
│   └── tools.py       # canonical tool surface (single source of truth)
├── scripts/
│   └── build_manifest.py   # regenerates plugin.json + the schema doc from tools.py
├── tests/             # unittest: dispatch-base unit + stdio round-trip + manifest
├── docs/
│   └── computer-use-mcp-tools-schema.md   # generated; the AC4 schema doc
├── run_tests.sh       # runs the suite (xvfb-wrapped when no DISPLAY)
├── pyproject.toml     # dependency-free, python>=3.10
├── README.md  LICENSE  .gitignore

Run it standalone

# speak MCP by hand (newline-delimited JSON-RPC):
printf '%s\n' \
  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"screenshot","arguments":{"save_to_disk":true}}}' \
  '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"zoom","arguments":{"region":[600,300,900,500]}}}' \
  '{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"type","arguments":{"text":"hello"}}}' \
  '{"jsonrpc":"2.0","id":6,"method":"tools/call","params":{"name":"key","arguments":{"text":"ctrl+a","repeat":1}}}' \
  '{"jsonrpc":"2.0","id":7,"method":"tools/call","params":{"name":"hold_key","arguments":{"text":"shift","duration":2}}}' \
  '{"jsonrpc":"2.0","id":8,"method":"tools/call","params":{"name":"left_click","arguments":{"coordinate":[600,300],"text":"ctrl"}}}' \
  | DISPLAY=:10 python3 src/server.py

initialize returns the handshake, tools/list the 24-tool surface, the screenshot call a base64 PNG image block (plus a Saved to disk: <path> text block when save_to_disk is set), and zoom a magnified PNG of the region. The keyboard and click calls each return a short text ack (isError:false); they need xdotool on PATH. The left_click maps its [600, 300] against the id:3 screenshot's coordinate session — a click before any screenshot returns a clear no screenshot session yet error instead of clicking blind.

Capture & coordinates

screenshot captures DISPLAY=:10 and, when the measured size matches a model resolution, downscales it (on the live 1920×1080 :10 it returns 1366×768). Each capture records a screenshot → coordinate session: the measured device geometry and the returned image geometry. Coordinates the model returns are in image space (relative to the last screenshot); the server maps them back to device pixels via Computer.to_device(x, y) — the foundation the click/move siblings (SCRUM-1401/1402) consume. Both the downscale and the coordinate mapping are derived from the same measured geometry, so they can never use different bases (the wrong-pixel-click failure mode).

zoom takes region: (x0, y0, x1, y1) in image space, maps it to a device rect, and crops it from a fresh full-resolution capture — genuine magnification, not an upscale of the downscaled screenshot. It is read-only: it never moves the click-coordinate origin (clicks still refer to the last screenshot). If no screenshot has been taken yet, zoom establishes the session lazily.

True 1:1 (no downscale) would require pinning :10 / the RDP window to a model-friendly size — that is provisioning (SCRUM-1396), out of scope here.

Click group (SCRUM-1401)

Pointer clicks at a screenshot-session coordinate. The [x, y] coordinate is image space (relative to the last screenshot) and is mapped to device pixels via Computer.to_device — a click before any screenshot returns a clear no screenshot session yet error (look before you click). Optional text modifier(s) ('ctrl', 'shift+alt', …) are held for the click and always released (keyup in a finally, the same guarantee as hold_key).

Tool

xdotool

Button

left_click

mousemove --sync <dx> <dy> click 1

left (1)

right_click

mousemove --sync <dx> <dy> click 3

right (3)

middle_click

mousemove --sync <dx> <dy> click 2

middle (2)

double_click

mousemove --sync <dx> <dy> click --repeat 2 --delay 100 1

left ×2

triple_click

mousemove --sync <dx> <dy> click --repeat 3 --delay 100 1

left ×3

With a modifier the click is wrapped in keydown -- <text> → click → keyup -- <text>. On-screen pixel accuracy is validated live in SCRUM-1408 (xdotool is absent on the current runner image, so the unit tests assert the device-pixel argv).

Keyboard group (SCRUM-1403)

Tool

xdotool

Notes

type

xdotool type --delay 12 -- <text>

types text at the current focus

key

xdotool key --repeat <repeat> -- <text>

chords, e.g. ctrl+s; optional repeat (default 1)

hold_key

keydown -- <text>sleep <duration>keyup -- <text>

the hold runs in the persistent server (Python time.sleep), so durations up to ~100s do not trip the per-call subprocess timeout; keyup runs in a finally so a held key/modifier is always released

Test

./run_tests.sh          # uses $DISPLAY if set, else wraps in xvfb-run
# or directly:
DISPLAY=:10 python3 -m unittest discover -s tests -v
  • tests/test_dispatch_base.py — coordinate-scaling math, the screenshot → coordinate session + to_device mapping, the measured-basis downscale target, zoom region validation + region→device-rect math, xdotool command construction, single-owner lock, screenshot-backend detection, surface shape, plus live screenshot/zoom + save_to_disk (DISPLAY-gated).

  • tests/test_stdio_roundtrip.pyinitializetools/listtools/call screenshot (incl. save_to_disk path block) and zoom over a spawned server, plus error paths.

  • tests/test_manifest.pyplugin.json + schema doc are present and in sync with src/tools.py.

The screenshot round-trip needs an X display; run_tests.sh provides one via xvfb-run when $DISPLAY is unset, so AC3 still exercises in headless CI.

System dependencies

System packages (apt), not pip — the plugin install copies no node_modules/venv and runs no build step, so the server is stdlib-only and shells out to:

Tool

Used for

Notes

a screenshot backend

screenshot

gnome-screenshot or scrot or ImageMagick (import/convert). The runner ships ImageMagick.

xdotool

pointer/keyboard actions; open_application fallback

required by the click/move/keyboard groups (siblings); absent on the runner image.

xclip

read_clipboard / write_clipboard

already on the runner; grant-gated.

wmctrl

open_application window focus

best-effort; open_application degrades to a no-op when absent.

scrot and gnome-screenshot are absent on the runner image, so the screenshot backend falls through to ImageMagick import -window root (verified on DISPLAY=:10). When SCRUM-1407 adds this plugin to the agent-runners catalog, declare xdotool, a screenshot backend, and xclip in its system_deps.

DISPLAY and single-owner

  • The server targets the inherited $DISPLAY, defaulting to :10 (the runner desktop). It never hardcodes a display — the screen-record plugin's bug was capturing a non-existent :1.0.

  • It takes a process-lifetime single-owner lock (flock, /tmp/sidebutton-computer-use.lock, override with CU_LOCK_PATH) so only one session drives the shared pointer/keyboard; a second instance exits non-zero.

Service-manifest contract (SCRUM-1406)

plugin.json targets the merged runtime: "service" tier: the SideButton server keeps the child alive, discovers its tools via tools/list, and forwards tools/call to it.

{
  "name": "computer-use",
  "runtime": "service",
  "service": {
    "command": "python3 src/server.py",  // non-empty string; the engine splits on
                                          // whitespace and spawns with cwd=plugin dir
    "toolNamespace": "computer_use",      // tools surface as computer_use_<tool>
    "tools": {                            // per-tool timeout overrides (ms)
      "hold_key": { "timeoutMs": 120000 },
      "wait":     { "timeoutMs": 120000 }
    }
  },
  "tools": []                             // service plugins declare no static tools
}

The loader (the-assistant packages/server/src/plugins/loader.ts) recognizes only command / timeoutMs / toolNamespace / tools under service, and hard-rejects the manifest unless command is a non-empty string — an array fails validation and the plugin never loads. Tools are discovered live, so the top-level tools array is normalized to []. This repo owns only plugin.json; the agent-runners catalog entry + system_deps are SCRUM-1407.

Configuration (env)

Var

Default

Purpose

DISPLAY

:10

target X display

CU_WIDTH / CU_HEIGHT

1920 / 1080

screen size for coordinate scaling

CU_SCREENSHOT_DELAY

2.0

post-action settle before a screenshot

CU_LOCK_PATH

/tmp/sidebutton-computer-use.lock

single-owner lock file

CU_SAVE_DIR

/tmp/sidebutton-computer-use/

where save_to_disk writes shareable PNGs (host-pruned; saved files are not auto-deleted)

License

MIT © 2026 SideButton

Available Tools

24 tools
computer_batchB

Run a sequence of computer-use actions in order and return one combined result.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionsYesOrdered list of {name, arguments} actions.

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description must disclose behavioral traits. It only states 'in order and return one combined result' but does not specify error handling, atomicity, or side effects like screen changes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no unnecessary words. It is front-loaded and efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

As a batch tool with no output schema, the description should explain the nature of the combined result and how errors are handled. It is incomplete for a tool that coordinates multiple actions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a description for the actions parameter. The tool description adds 'in order', which reinforces the schema, but does not add significant new meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'run' and the resource 'sequence of computer-use actions', with the result 'one combined result'. This distinguishes it from individual action tools like left_click or type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies batch usage but provides no explicit guidance on when to use this tool versus calling individual actions sequentially. It does not mention prerequisites or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

cursor_positionA

Return the current cursor [x, y] in model coordinates.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes exact return format and coordinate system. No annotations exist, but the description covers the core behavior adequately. Could mention that it is non-destructive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence with no extraneous information. Front-loaded with key action and output.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complete for a zero-parameter query tool. Output format is specified. No output schema needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters, so schema fully covers. Description adds value by specifying 'model coordinates' for the output.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clear verb 'Return' and specific resource 'current cursor [x, y]' in model coordinates. Distinct from sibling action tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use versus alternatives. Implicitly a read-only query, but no when-not or context provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

double_clickB

Double left-click at a coordinate.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateYes[x, y] in the model coordinate space (scaled to the screen).
textNoOptional modifier key(s) to hold during the click, e.g. 'ctrl' or 'shift+alt'.

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must fully disclose behavior. It only states the basic action without mentioning nuances like whether it performs a full press-release-press-release sequence, any inherent delays, or side effects. This is insufficient for an agent to predict the tool's impact.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys the core action. No redundant information is present, making it optimally concise for a straightforward tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity and the absence of an output schema, the description is moderately complete. It defines the action and parameters, but lacks context about the typical behavior of double-clicking (e.g., triggering default actions) which would help the agent choose this tool over siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides complete descriptions for both parameters (coordinate and text). The tool description adds no additional meaning beyond restating 'Double left-click at a coordinate.' Since schema coverage is 100%, a baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Double left-click at a coordinate.' This distinguishes it from siblings like left_click or triple_click by specifying the double-click action. However, it doesn't elaborate on what double-clicking does in context, missing a chance to differentiate further.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. For example, when a double-click is preferred over a single click or triple click, or that it might be used to open files or invoke default actions. The agent receives no contextual cues.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

hold_keyC

Hold a key (or chord) down for a duration in seconds.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes
durationYesSeconds to hold (may be up to ~100s).

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description only says 'Hold down' but does not clarify whether the key is automatically released after the duration, how chords are specified (e.g., separator), or any side effects. With no annotations, more detail is needed for safe usage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, front-loaded with key information. It is concise but could include more detail without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 2 required parameters, no output schema, and no annotations, the description is insufficient. It omits the required format for 'text', behavior after duration, and any constraints, making it incomplete for an agent to use correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 50% (only duration has a description). The description adds no meaning to the 'text' parameter (e.g., format for chords) beyond the schema. The tool description does not compensate for the missing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Hold a key (or chord) down for a duration in seconds' explicitly states the verb (hold), resource (key or chord), and duration, clearly distinguishing it from sibling tools like 'key' or 'type' which likely involve pressing and releasing or typing characters.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use hold_key versus alternatives such as 'key' or 'type'. The description does not indicate context, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

keyA

Press a key or chord using xdotool key syntax, e.g. 'Return', 'ctrl+s', 'alt+Tab'.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, and the description only says 'press', failing to disclose whether it presses and releases, any prerequisites (e.g., window focus), or potential side effects. The behavioral detail is minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with examples, containing no redundant information. Every part earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool, the description covers purpose and syntax adequately but lacks information on return values or system behavior (e.g., whether it works globally). Without an output schema, some completeness is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The sole parameter 'text' has 0% schema coverage, but the description adds significant meaning by specifying xdotool key syntax and providing examples, effectively guiding the agent on parameter formatting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the tool presses a key or chord using xdotool key syntax, with concrete examples ('Return', 'ctrl+s', 'alt+Tab'), making the purpose clear and distinct from siblings like 'hold_key' or 'type'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Examples imply usage but no explicit guidance on when to use this tool over alternatives like 'hold_key' or 'type'. It does not specify context or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

left_clickC

Left-click at a coordinate.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateYes[x, y] in the model coordinate space (scaled to the screen).
textNoOptional modifier key(s) to hold during the click, e.g. 'ctrl' or 'shift+alt'.

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must disclose behavioral traits. It only states the action without effects, prerequisites, or outcomes (e.g., whether the cursor moves, if the click is instant, or if it can fail). This is insufficient for agent understanding.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise—one front-loaded sentence with no wasted words. It efficiently conveys the core action.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of output schema and annotations, the description should explain what happens after execution (e.g., return value, success/failure behavior). It does not, leaving an incomplete picture for the agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds no parameter meaning beyond the schema, which already explains coordinate as [x,y] in model space and text as optional modifier keys.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it performs a left-click at a coordinate. It specifies the verb 'click' and the resource 'coordinate', but does not distinguish from siblings like left_click_drag or double_click, which also involve left-clicking.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives (e.g., left_click_drag, double_click). There is no mention of context or exclusions, leaving the agent without direction for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

left_click_dragA

Press the left button at start_coordinate (or the current position) and drag to coordinate before releasing.

ParametersJSON Schema
NameRequiredDescriptionDefault
start_coordinateNo[x, y] in the model coordinate space (scaled to the screen).
coordinateYes[x, y] in the model coordinate space (scaled to the screen).

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It describes the basic sequence (press, drag, release) but lacks details about timing, acceleration, or what happens if start_coordinate is omitted. A more thorough description would mention edge cases or potential side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It front-loads the essential action and uses parentheses for optional behavior, which is well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, 100% schema coverage, and lack of output schema, the description is largely complete. It explains the start and end points. However, it could mention that this is a mouse drag action (e.g., for moving objects) to improve intuitive understanding.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Both parameters have schema descriptions with coordinate format details. The tool description adds value by noting that start_coordinate is optional and defaults to the current position, which is not fully captured in the schema alone (schema makes coordinate required but doesn't clarify start_coordinate's role).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: pressing the left button at start_coordinate (or current position) and dragging to a target coordinate. It uses a specific verb and resource, distinguishing it from related tools like left_click, mouse_move, and left_mouse_down.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for drag operations but does not explicitly state when to use this tool versus alternatives like left_mouse_down + mouse_move + left_mouse_up or other click-and-drag methods. No guidance on prerequisites or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

left_mouse_downA

Press and hold the left mouse button (released later by left_mouse_up). Requires the persistent session.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateNo[x, y] in the model coordinate space (scaled to the screen).

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It clearly discloses the hold behavior and that release requires left_mouse_up, which is key. The persistent session requirement is also stated. Could mention potential issues if already pressed, but overall adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that efficiently conveys the action and the prerequisite. No unnecessary words or details. Perfectly front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no output schema, the description covers the essential behavior, coordinate meaning, and session requirement. It also references the companion tool for release. Minor omission: no mention of constraints like coordinate bounds or timeout.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides a complete description of the coordinate parameter (format, constraints, meaning). The description adds no further semantic value beyond what the schema states, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'Press and hold the left mouse button' with a clear verb and resource. It distinguishes from sibling tools like left_click (press and release) and left_click_drag (press, move, release) by noting the hold action and release via left_mouse_up.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when a persistent mouse press is needed, but no explicit when-to-use or when-not-to-use guidance is given. It mentions the persistent session requirement but does not contrast with alternatives like left_click or left_click_drag.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

left_mouse_upB

Release a left mouse button held by left_mouse_down.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateNo[x, y] in the model coordinate space (scaled to the screen).

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It does not disclose behavioral traits such as what happens if the button is not held, effects of the coordinate parameter, or any side effects beyond the release action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Very concise single sentence, but minimal structure. Adequate for a simple tool, but lacks any additional context that could help agent understanding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool is simple and has no output schema, the description could still clarify the relationship with left_mouse_down and coordinate usage. Missing details reduce completeness for confident invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter 'coordinate', so baseline 3 applies. The description does not add additional meaning beyond what is already in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('release') and the specific resource ('left mouse button held by left_mouse_down'), using a specific verb+resource. It implicitly distinguishes from sibling tools like left_mouse_down and left_click.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool instead of alternatives. It only mentions the relationship to left_mouse_down but does not specify context like required prior actions or error handling.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_granted_applicationsA

Return the set of applications currently granted desktop access (Linux stub: echoes the granted set).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden. It discloses the read-only nature and includes a note about Linux behavior. However, it lacks details on side effects, permissions, or what constitutes 'desktop access'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no fluff. It is appropriately sized and front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with no parameters and no output schema, the description covers the basic functionality. It mentions the Linux stub behavior, but could be improved by clarifying the scope of 'desktop access'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so schema coverage is 100%. Baseline for 0 parameters is 4. The description does not need to add parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Return') and the resource ('set of applications currently granted desktop access'). It effectively distinguishes from sibling tools like 'request_access' and 'open_application', which involve different operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives such as 'request_access' or 'open_application'. The description only states what it does without usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

middle_clickC

Middle-click at a coordinate.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateYes[x, y] in the model coordinate space (scaled to the screen).
textNoOptional modifier key(s) to hold during the click, e.g. 'ctrl' or 'shift+alt'.

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries full burden. It discloses only that a middle-click occurs at a coordinate, but not whether it simulates a real click, any side effects, or system requirements. The modifier key parameter description adds minor transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise—a single sentence with no unnecessary words. It is efficiently front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the large set of sibling click tools and no output schema, the description is too minimal. It does not explain typical use cases (e.g., opening links in a new tab) or return behavior, leaving the agent without sufficient context for correct selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters having descriptions. The tool-level description adds no extra parameter meaning, but the schema already provides adequate explanation (e.g., coordinate format, modifier keys). Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (middle-click) and the target (at a coordinate), distinguishing it from other click tools like left_click or right_click. However, it is minimal.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use middle-click versus alternatives (e.g., left_click, double_click). Usage is only implied, with no exclusions or context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_moveA

Move the pointer to a coordinate without clicking.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateYes[x, y] in the model coordinate space (scaled to the screen).

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden. It discloses that the tool moves without clicking, but omits information about animation, boundary behavior, or side effects. Minimal but not misleading.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, 10 words, front-loaded with the verb and resource. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple move tool with one parameter and no output schema, the description is minimally adequate. It does not explain the coordinate space scaling or potential restrictions, but the schema covers the parameter.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with coordinate described as [x,y] in model coordinate space. The description adds no further parameter meaning beyond the schema, aligning with the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (move), the resource (pointer), and a specific constraint (without clicking). This distinguishes it from clicking siblings like left_click.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives such as left_click_drag or scroll. With 22 sibling tools, the description should provide usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

open_applicationB

Launch or focus a desktop application by name.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It mentions 'launch or focus' which implies behavior, but does not disclose error handling (e.g., app not found), permissions, or side effects like potential blocking.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, front-loaded with action, no wasted words. Appropriate size for the tool's simplicity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, no annotations, and a single parameter, the description lacks details on return values or error conditions. An agent cannot determine success or failure handling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description only says 'by name', which adds minimal context beyond the parameter name. Does not specify format, case sensitivity, or possible values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool launches or focuses a desktop application by name, providing a specific verb and resource. It distinguishes from sibling tools like 'list_granted_applications' which lists apps rather than launching them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives (e.g., when the application might already be running). No exclusions or prerequisites mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_clipboardA

Read the X clipboard contents (via xclip).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. Description mentions the underlying tool (xclip) but does not disclose behavior on empty clipboard, missing xclip, or side effects. Minimal transparency for a simple read operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no wasted words, directly conveys purpose and mechanism. Excellent conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Simple tool with no output schema or parameters. Description mentions the specific clipboard source (X clipboard) and tool (xclip). Lacks detail on return format but adequate for a straightforward read.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters exist, so schema coverage is 100%. Description adds no param info, which is acceptable per guidelines. Baseline score of 4 for zero-parameter tools.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Read the X clipboard contents (via xclip)', providing a specific verb, resource, and mechanism. It effectively distinguishes from sibling 'write_clipboard'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives (e.g., when to read vs write clipboard). No context about prerequisites or situations where reading might fail.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

request_accessB

Request a session grant for one or more applications (Linux stub: auto-grants and returns screenshotFiltering=false; the real grant model lands with the service engine).

ParametersJSON Schema
NameRequiredDescriptionDefault
applicationsNoApplications to request access to.

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries full burden. It discloses the Linux stub behavior and future model, but lacks details on non-Linux platform behavior, error handling, or side effects. Partially transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One concise sentence with a parenthetical that adds important context. No fluff, but the parenthetical might be slightly awkward. Good front-loading.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and one parameter, the description covers Linux stub behavior but omits essential context like non-Linux behavior, error conditions, grant lifecycle, and usage on other platforms. Incomplete for an access-granting tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a single parameter 'applications' described as 'Applications to request access to.' The description adds no additional meaning beyond 'one or more applications,' which is already clear from the schema. Baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: to request a session grant for applications. It provides specific verb ('Request'), resource ('session grant'), and distinguishes from siblings like list_granted_applications by mentioning Linux stub behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives. It describes Linux stub behavior but does not specify prerequisites, when not to use, or differences from sibling tools like open_application or list_granted_applications.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

right_clickC

Right-click at a coordinate.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateYes[x, y] in the model coordinate space (scaled to the screen).
textNoOptional modifier key(s) to hold during the click, e.g. 'ctrl' or 'shift+alt'.

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only states the action without explaining behavioral traits like whether it requires focus, what coordinate system is used (though the param schema notes model space), or what effects occur (e.g., opening context menu). The description adds no behavioral context beyond the name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise at just 4 words, but it is appropriately structured for a simple action. However, it could be more informative without becoming verbose, so it scores average.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity and complete schema coverage, the description is still too terse. It does not explain the return value (e.g., success indicator or new cursor position) or how it integrates with other tools (e.g., requiring prior mouse movement). This leaves the agent with incomplete information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters: coordinate as [x,y] in model space and text as optional modifier keys. The tool description does not add any additional meaning or usage nuances beyond what the schema provides, resulting in a baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it performs a right-click at a coordinate. The verb 'right-click' and the resource 'coordinate' are specific. However, it does not differentiate from sibling tools like left_click or double_click, which have similar structures.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like left_click or triple_click. The description does not mention any context or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screenshotB

Capture the active display (DISPLAY=:10) and return a base64-encoded PNG. Implemented in SCRUM-1397 as the proof action.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. Description only specifies the display (DISPLAY=:10) and adds an implementation reference, but fails to disclose permissions, failure modes, or effects like whether it captures full screen or window.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences; the first is essential and clear. The second sentence adds implementation context but is not critical. Still concise overall.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, so description should provide more detail on the returned data (e.g., size, details) or behavior (e.g., asynchronous, permissions). States only output format. Incomplete for a tool with zero annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

No parameters, so baseline 4. The description already states the output format, which is useful beyond the empty schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states verb 'capture', resource 'active display', and output format 'base64-encoded PNG'. Distinguishes from sibling tools that perform other actions like clicking or moving.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool vs alternatives. Does not mention when not to use or provide context for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scrollC

Scroll in a direction by an amount at a coordinate.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateYes[x, y] in the model coordinate space (scaled to the screen).
scroll_directionYes
scroll_amountYesNumber of scroll 'clicks'.

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Lacks details on effects of off-screen coordinates, scroll amount scaling, or side effects. Annotations absent, so description carries full burden but falls short.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence is concise and front-loaded, but slightly too brief for a tool with three parameters.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Complete for a trivial scroll action but omits crucial details like coordinate system, units, and limitations, leaving room for misuse.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage 67%; description adds no extra meaning. The scroll_direction enum lacks any explanation, and coordinate/scroll_amount descriptions are already in schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action 'scroll' with direction, amount, and coordinate, distinguishing it from sibling mouse tools like click or move.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use scroll versus alternatives like mouse_move or click-drag. No prerequisites or context provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

switch_displayC

Switch the display the session targets, e.g. ':10'.

ParametersJSON Schema
NameRequiredDescriptionDefault
displayYes

TDQS

C2.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description is responsible for disclosing behavior. It only states the action and an example, but does not mention effects on the session, safety (destructive or not), or any side effects. This is insufficient for a mutation-like tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short (one sentence), but it omits critical information, making it under-specified rather than concise. Every sentence should add value; here it fails to provide sufficient context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity (no nested objects, no output schema), the description should be complete. It is not: missing parameter format, usage context, and behavioral details. The tool's job is clear but the description is incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must explain the parameter. It gives an example (':10') but does not describe the format, valid values, or how the 'display' string is interpreted. The agent lacks needed semantic detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action ('switch the display') and provides an example format (':10'), making the purpose easy to understand. However, the term 'session targets' could be more explicit, but it's still adequate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus the many sibling tools (e.g., mouse_move, scroll). The description does not mention prerequisites or conditions for usage, leaving the agent to infer context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

triple_clickB

Triple left-click at a coordinate.

ParametersJSON Schema
NameRequiredDescriptionDefault
coordinateYes[x, y] in the model coordinate space (scaled to the screen).
textNoOptional modifier key(s) to hold during the click, e.g. 'ctrl' or 'shift+alt'.

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided; the description is minimal and does not disclose any behavioral traits beyond the bare action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded; every word is necessary, but could be slightly more structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity and no output schema, the description is minimally complete but lacks guidance on behavior or use cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3; the description adds no additional meaning beyond what the schema already provides for the parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Triple left-click at a coordinate' clearly specifies the action and target, distinguishing it from siblings like double_click and left_click.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use triple click versus alternatives; the description only states the action without context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

typeB

Type a string of text at the current focus.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since no annotations are provided, the description must fully disclose behavior. It mentions the tool types 'at the current focus', but does not specify what happens if no element is focused, how special characters are handled, or whether the action is destructive. Additional context about keystroke simulation or speed is missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no extraneous information. Every word is necessary and contributes to understanding the tool's purpose. It is appropriately concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description provides the core functionality. However, it lacks edge-case handling (e.g., no focus), behavioral details, and usage context. It is minimally adequate but not thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'text' has no schema description (0% coverage), so the description must compensate. It describes it as 'a string of text', which adds little beyond the schema. No details on format, length limits, or allowed characters are provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Type' and the resource 'a string of text', with context 'at the current focus'. This distinguishes it from sibling tools like 'key' (single key presses) and 'write_clipboard' (clipboard operations).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as 'key' for individual keystrokes or 'hold_key' for modifier keys. The description does not mention scenarios or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

waitA

Wait for a duration in seconds (then optionally screenshot).

ParametersJSON Schema
NameRequiredDescriptionDefault
durationYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses the primary behavior (wait for duration, optionally screenshot) but lacks details on blocking, error handling, or edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no excess. Every word contributes to understanding the tool's function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description covers the essential behavior and optional screenshot. It could mention return behavior (likely none) but is largely complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds value by specifying 'in seconds' for the duration parameter, which the schema (number, required) does not provide. This clarifies the unit and usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'wait' and the resource 'duration in seconds', with an optional screenshot action. It distinguishes itself from sibling tools as the only wait-related tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use or avoid the tool. No mention of alternatives or prerequisites, leaving the agent to infer usage from context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

write_clipboardB

Write text to the X clipboard (via xclip).

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the action and context but does not specify that it overwrites the current clipboard content, nor does it mention permissions or side effects. With no annotations, more behavioral detail is expected.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that is clear and front-loaded. Every word serves a purpose with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description is mostly adequate. However, it lacks usage guidelines and parameter elaboration, leaving some gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With schema coverage at 0%, the description does not add any meaning to the 'text' parameter beyond the schema. It could clarify length limits, encoding, or format but does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Write', the resource 'text to the X clipboard', and the mechanism 'via xclip'. It immediately distinguishes from the sibling 'read_clipboard' tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like 'type' or 'key'. No mention of when not to use it or prerequisite conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

zoomB

Capture and return a magnified PNG of a sub-region of the screen.

ParametersJSON Schema
NameRequiredDescriptionDefault
regionYes[x, y, width, height] region to magnify.

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It states the output is a 'magnified PNG' but does not disclose the magnification factor, potential permissions, or whether the region is relative to the screen or application. Insufficient detail for safe invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that is clear and free of superfluous content. Every word contributes to understanding the tool's function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema, no annotations), the description covers the basic function. However, it omits important details like the output format (file path vs. base64), magnification factor, and how to interpret coordinates. A 3 reflects that it is minimally adequate but has clear gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the 'region' parameter, so the baseline is 3. The description adds no extra meaning beyond the schema, only reiterating the purpose. It does not clarify coordinate system or constraints.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Capture and return a magnified PNG') and the resource ('sub-region of the screen'). The verb 'Capture' and modifier 'magnified' differentiate it from the sibling 'screenshot' tool, but it could be more explicit about the distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'screenshot' or 'mouse_move'. There are no exclusions or contextual cues for appropriate usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 24 tool updatesv0.1.0
    • First observedcomputer_batch
    • First observedcursor_position
    • First observeddouble_click
    • First observedhold_key
    • First observedkey
    • First observedleft_click
    • First observedleft_click_drag
    • First observedleft_mouse_down
    • First observedleft_mouse_up
    • First observedlist_granted_applications
    • First observedmiddle_click
    • First observedmouse_move
    • First observedopen_application
    • First observedread_clipboard
    • First observedrequest_access
    • First observedright_click
    • First observedscreenshot
    • First observedscroll
    • First observedswitch_display
    • First observedtriple_click
    • First observedtype
    • First observedwait
    • First observedwrite_clipboard
    • First observedzoom

TDQS

B3.2/5.0

Scored across 24 tools

Disambiguation4/5

Most tools have distinct purposes like left_click vs right_click vs double_click, but some overlap exists (e.g., left_click_drag vs left_mouse_down/move/up sequence) and could cause confusion despite clear descriptions.

Naming Consistency3/5

Naming is mostly snake_case but inconsistent in verb_noun patterns: some are verbs (type, scroll), some nouns (screenshot, key), and some are compound (computer_batch). Not chaotic but lacks a uniform style.

Tool Count4/5

24 tools is reasonable for a computer-use server covering mouse, keyboard, clipboard, screenshots, and display management. Not too many, not too few.

Completeness4/5

Covers core desktop automation actions well (clicks, drag, keyboard, clipboard, screenshots, scrolling, application launching). Minor gaps like file operations or advanced gestures are acceptable.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    B
    maintenance
    An MCP server that gives any AI assistant eyes and hands on your desktop — screenshots, clicking, typing, OCR, window management, accessibility-tree queries, workflow recording.
    5
    Apache 2.0
  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables Windows desktop automation via MCP, allowing AI agents to control mouse, keyboard, and screen capture with the same interface as Anthropic's computer-use tool.
    3
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    A framework-agnostic computer-use MCP server that exposes core desktop operations (screen capture, mouse, keyboard, and file access) as standard MCP tools, enabling any MCP-compatible agent to drive a computer.
    347
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/sidebutton/plugin-computer-use'

If you have feedback or need assistance with the MCP directory API, please join our Discord server