Skip to main content
Glama

sandbox_click

Clicks at a given screen coordinate in a sandbox VM to operate graphical programs. Take a screenshot first to find buttons or fields, then click and verify the result.

Instructions

Click the mouse at a position on a sandbox's screen.

Use this to operate graphical programs: press buttons, open menus, focus a text field before typing. First take a sandbox_screenshot, find the thing to click, and pass its position: x pixels from the left edge and y pixels from the top edge of that image. Take another screenshot afterwards to see the result; screens need a moment to react.

button: "left" (default), "right" or "middle". double: true for a double-click. Only works if the user has switched keyboard and mouse input on; the error says so otherwise. For anything that can be done with a command, sandbox_run is faster and more reliable.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "sandbox_id": "<sandbox_id>",
      "x": "<x>",
      "y": "<y>"
    },
    "name": "sandbox_click"
  }
}

Input Schema

NameRequiredDescriptionDefault
xYes
yYes
buttonNoleft
doubleNo
sandbox_idYes

Output Schema

NameRequiredDescriptionDefault
resultYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden and does well: it discloses the input-enabled prerequisite and how failure surfaces ('the error says so otherwise'), plus the latency caveat ('screens need a moment to react'). It does not mention coordinate bounds handling or what a click on an obscured element does, leaving minor gaps for a UI-mutating action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded action sentence, then procedure, then parameter notes, then the alternative-tool caveat. Every sentence carries usable information, including the practical advice about waiting for the screen to react.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a five-parameter UI-action tool with no annotations, the description covers purpose, prerequisites, sequencing, parameter meanings, and the preferred alternative. An output schema exists, so the absence of return-value detail is not a gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema coverage the description must compensate, and it does for the hard parameters: it defines x/y as pixels from the left/top edge of the screenshot image, enumerates button values with the left default, and explains double=true. sandbox_id is never described, and no bounds/format detail is given for the coordinates.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb and resource: 'Click the mouse at a position on a sandbox's screen.' It immediately scopes the tool to graphical interaction and names the sibling it competes with ('sandbox_run is faster and more reliable'), so an agent can distinguish it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit when-to-use ('press buttons, open menus, focus a text field before typing'), a sequencing procedure (screenshot → click → screenshot), a prerequisite (user must enable keyboard/mouse input), and a named alternative (sandbox_run) with the condition that selects it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.