Skip to main content
Glama
Harusame64

desktop-touch-mcp

by Harusame64
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

workspace_snapshot

Capture all display layouts, window thumbnails, and per-window actionable UI elements with click coordinates in one call to orient before acting.

Instructions

Purpose: Orient fully in one call — returns display layouts, all window thumbnails (WebP), and per-window actionable element lists with clickAt coords. Details: uiSummary.actionable[] per window includes: action ('click'|'type'|'expand'|'select'), clickAt {x,y} (pass directly to mouse_click), value (current text for editable fields). Runs parallel internally; latency ≈ max(single screenshot), not N×screenshots. Also resets the diffMode buffer so subsequent screenshot(diffMode=true) returns only changes (P-frame). Prefer: Use at session start or after major workspace changes. Use screenshot(detail='meta') for cheap re-orientation within a session. Use screenshot(detail='text', windowTitle=X) for a single-window update. Caveats: Thumbnails are scaled, not 1:1 — use screenshot(dotByDot=true, windowTitle=X) for pixel-accurate coords on a specific window after snapshot. Also: this call resets the screenshot diff baseline (I-frame) and identity tracker as a side effect, so subsequent screenshot(diffMode=true) starts fresh from this snapshot. The reset is not currently exposed in causal/working memory — record an explicit 'workspace_snapshot' step if you need to track the reset point in your causal trail (ADR-010 §11 OQ carry-over for full visibility).

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
includeNoOptional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients).
includeUiSummaryNoWhether to include UI element summaries for each window
thumbnailMaxDimensionNoMax size of per-window thumbnail images (default 400px)

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed4 schema fields changedv2.3.1
    • addedInput schema / $schema
      Added value: +"http://json-schema.org/draft-07/schema#"
    • removedInput schema / additionalProperties
      Removed value: -false
    • addedInput schema / properties / thumbnailMaxDimension / exclusiveMinimum
      Added value: +0
    • addedInput schema / properties / thumbnailMaxDimension / maximum
      Added value: +9007199254740991
  2. Addedv1.9.2
  3. Removedv1.8.0
  4. Addedv1.6.0
  5. Removedv1.5.1
  6. Addedv1.5.0
  7. Removedv1.4.3
  8. Addedv1.4.2
  9. Removedv1.4.0
  10. Changed1 schema field changedv1.2.1
    • addedInput schema / properties / include
      Added value: +{
+  "description": "Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients).",
+  "items": {
+    "type": "string"
+  },
+  "type": "array"
+}
  11. Addedv1.1.1
  12. Removedv1.1.0
  13. Addedv1.0.3
  14. Removedv0.15.7
  15. Changed5 schema fields changedv0.10.0
    • removedInput schema / $schema
      Removed value: -"http://json-schema.org/draft-07/schema#"
    • addedInput schema / additionalProperties
      Added value: +false
    • removedInput schema / properties / _stub
      Removed value: -{
-  "description": "Stub tool — arguments ignored."
-}
    • addedInput schema / properties / includeUiSummary
      Added value: +{
+  "default": true,
+  "description": "Whether to include UI element summaries for each window",
+  "type": "boolean"
+}
    • addedInput schema / properties / thumbnailMaxDimension
      Added value: +{
+  "default": 400,
+  "description": "Max size of per-window thumbnail images (default 400px)",
+  "type": "integer"
+}
  16. First observedv0.6.2

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description carries the full burden and does so well: it discloses internal parallelism, latency characteristics (≈ max single screenshot), scaled-not-1:1 thumbnails, and a significant side effect (resetting the screenshot diff baseline/identity tracker). It even flags that the reset isn't reflected in causal memory, which is rare and useful context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Well front-loaded with labeled sections (Purpose, Details, Prefer, Caveats) that let an agent scan quickly. However, it is verbose and the diff-baseline reset is stated twice (once as 'resets the diffMode buffer', again as 'resets the screenshot diff baseline'), and jargon like 'ADR-010 §11 OQ carry-over' adds length without actionable value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly explains the return shape (layouts, thumbnails, actionable element lists with coords) and covers the side effects and caveats an agent needs before calling. Nothing material is missing for a zero-required-parameter orientation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the three input parameters (include, includeUiSummary, thumbnailMaxDimension) are already fully documented in the schema. The description adds no input-parameter semantics beyond that; its detailed field talk (clickAt, actionable[]) describes response content, not inputs. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening line states a specific verb ('Orient fully') and enumerates exactly what is returned: display layouts, WebP thumbnails, and per-window actionable element lists with clickAt coords. It is clearly distinguishable from screenshot siblings by naming the alternatives (screenshot(detail='meta'), screenshot(detail='text')) and contrasting their scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly says when to use it ('session start or after major workspace changes') and routes each alternative case to the correct sibling tool ('screenshot(detail=meta)' for cheap re-orientation, 'screenshot(detail=text, windowTitle=X)' for a single window). This is textbook when/when-not/alternative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools