Skip to main content
Glama

Robot Actions — Remote Device Control

by com.robotactions.test
OverviewDiscussionsAdmin

Device tap

device_tap

Tap at (x,y) coordinates on a device screen. Coordinates are in DEVICE TAP-COORD SPACE (the "Tap-coord space" dims printed in the device_screenshot footer; same space as device_page_source bounds). First call starts a device control session (~3s).

COORDINATE SOURCES — in priority order:

  1. PRIMARY: device_page_source bounds [L,T][R,B] (or the "Labeled elements" block bundled with device_screenshot) → tap center = ((L+R)/2, (T+B)/2). NO scaling. Pixel-exact.

  2. FALLBACK ONLY (element not in page_source — image-only widget / custom Canvas): visual estimate from the screenshot pixels, scaled with the formula below.

VISUAL → TAP COORDINATE FORMULA (Android): scale = device_width / rendered_chat_width tap_x = visual_x × scale tap_y = visual_y × scale where device_width is the "Tap-coord space" width from the device_screenshot footer and rendered_chat_width is the "Image" width from the same footer. Both axes share one scale (aspect preserved). The footer prints concrete values per device — never assume any constant.

Skipping the scale on a visual estimate is the #1 cause of taps landing in the wrong place — the agent sees a downscaled image but device_tap expects full-resolution tap-space coords.

Pass input="hid" to require the virtual touchscreen: the gesture then reaches the app as real kernel input and fails loudly instead of silently falling back to injected input. That strict path used to be device_uhid_tap, which stays callable under its old name.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
xYesX coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)
yYesY coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)
udidYesDevice serial number (UDID)
inputNoInput path. 'auto' (default) prefers the virtual touchscreen and falls back to injected input. 'hid' requires the virtual touchscreen and fails if the device does not provide one — use it when the gesture must be real kernel input or must fail loudly.
holdMsNoHow long the contact stays down, in ms. Only applies with input="hid" (default 60ms).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed3 schema fields changed
    • changedInput schema / properties / input / description
      Previous value: -"Input path. 'auto' (default) prefers the virtual touchscreen and falls back to injected input. 'hid' requires the virtual touchscreen and fails if the device does not provide one — use it when the gesture must be real kernel input or must fail loudly."New value: +"Input path. 'auto' (default) prefers the virtual touchscreen and falls back to injected input. 'hid' requires the virtual touchscreen and fails if the device does not provide one — use it when the gesture must be real kernel input or must fail loudly."
    • changedInput schema / properties / x / description
      Previous value: -"X coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"New value: +"X coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"
    • changedInput schema / properties / y / description
      Previous value: -"Y coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"New value: +"Y coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"
  2. Changed3 schema fields changed
    • changedInput schema / properties / input / description
      Previous value: -"Input path. 'auto' (default) prefers the virtual touchscreen and falls back to injected input. 'hid' requires the virtual touchscreen and fails if the device does not provide one — use it when the gesture must be real kernel input or must fail loudly."New value: +"Input path. 'auto' (default) prefers the virtual touchscreen and falls back to injected input. 'hid' requires the virtual touchscreen and fails if the device does not provide one — use it when the gesture must be real kernel input or must fail loudly."
    • changedInput schema / properties / x / description
      Previous value: -"X coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"New value: +"X coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"
    • changedInput schema / properties / y / description
      Previous value: -"Y coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"New value: +"Y coordinate in device tap-coord space (NOT visual screenshot pixels — apply scale = device_width / rendered_chat_width if you started from a visual estimate)"
  3. Changed2 schema fields changed
    • addedInput schema / properties / holdMs
      Added value: +{
+  "description": "How long the contact stays down, in ms. Only applies with input=\"hid\" (default 60ms).",
+  "maximum": 5000,
+  "minimum": 1,
+  "type": "integer"
+}
    • addedInput schema / properties / input
      Added value: +{
+  "description": "Input path. 'auto' (default) prefers the virtual touchscreen and falls back to injected input. 'hid' requires the virtual touchscreen and fails if the device does not provide one — use it when the gesture must be real kernel input or must fail loudly.",
+  "enum": [
+    "auto",
+    "hid"
+  ],
+  "type": "string"
+}
  4. First observed

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare non-read-only, non-destructive, non-idempotent, but the description adds substantial context the annotations cannot: the ~3s session startup on first call, the strict-failure semantics of input="hid", and the constraint that holdMs only applies with hid. This is real behavioral disclosure, not restated annotation data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and coordinate space, then structured into clearly labeled coordinate sources and a formula. Dense but mostly earning its space; the trailing aside about the former device_uhid_tap name is historical trivia that adds little.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description covers coordinate derivation, session startup, and input-path behavior thoroughly. It omits what a failed tap returns or how errors surface, a minor gap given annotations already cover the safety profile.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is already 100%, so the baseline is 3. The description exceeds that by supplying the coordinate-source priority order and the Android scaling formula rationale (the '#1 cause of taps landing in the wrong place'), which adds operational meaning beyond the per-parameter schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Tap at (x,y) coordinates on a device screen') and pins down the exact coordinate space, which distinguishes it from device_tap_by_text, device_long_press, and device_two_finger_tap without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit priority order for coordinate sources (page_source bounds first, visual estimate only as fallback) and explains the hid vs auto input path. It does not, however, say when to prefer this tool over siblings like device_tap_by_text or device_find_element, so routing guidance is incomplete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All Robot Actions — Remote Device Control toolsInspect
Try in Browser
Let your AI agent see the webWeb-to-screenshot MCP for Claude, Cursor and Codex. No cookie banners. Free to start.snap-render.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources