Skip to main content
Glama

glass_do

Execute a fixed sequence of UI actions—clicks, typing, scrolling, waits—then run terminal observations like screenshots or diffs. Stops on the first error to prevent cascading failures.

Instructions

Batch 2+ known actions or waits here; observe before choosing dependent steps. Run one action or fixed static ordered actions: click, move, drag, scroll, type, key, settle, click_element, set_value, wait_for_element, scroll_to_element. Maximum 64 actions and 65536 compact argument bytes. timeout_ms defaults to 30000; range 1 through 120000; one deadline includes terminal observations. Fail-fast on action errors, deadline or unmatched batched predicates; standalone predicates remain soft. Success without then returns ordered steps with result and content_blocks; ok:true means all actions completed. Failures and then retain completed, failed, unexecuted and terminal_steps; inspect before recovery. Preflight invalid_sequence has no step outcomes. Terminal settle/diff/screenshot runs after actions; terminal failure retains completed action outcomes. No branching, bindings, loops, retries or generated steps. Semantic targets resolve fresh; targeted type requires confirmed focus; set_value requires confirmation. Never replay completed or possibly dispatched mutations; observe after uncertainty.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
thenNoTerminal observation in order: settle, diff, screenshot.
actionsYesNon-empty ordered actions; failure stops remaining steps. Completed mutations may already have landed.
timeout_msNoOverall budget in ms (default 30000, range 1..120000), shared by all actions and terminal observations.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed74 schema fields changedv1.8.0
    • addedInput schema / $defs / ActionModeArg
      Added value: +{
      +  "enum": [
      +    "auto",
      +    "native",
      +    "pointer"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / $defs / ActionScopeArgs
      Added value: +{
      +  "additionalProperties": false,
      +  "properties": {
      +    "query": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "role": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "states": {
      +      "items": {
      +        "type": "string"
      +      },
      +      "type": [
      +        "array",
      +        "null"
      +      ]
      +    }
      +  },
      +  "type": "object"
      +}
    • addedInput schema / $defs / ActionTargetArgs
      Added value: +{
      +  "additionalProperties": false,
      +  "properties": {
      +    "query": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "role": {
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "states": {
      +      "items": {
      +        "type": "string"
      +      },
      +      "type": [
      +        "array",
      +        "null"
      +      ]
      +    },
      +    "within": {
      +      "anyOf": [
      +        {
      +          "$ref": "#/$defs/ActionScopeArgs"
      +        },
      +        {
      +          "type": "null"
      +        }
      +      ]
      +    }
      +  },
      +  "type": "object"
      +}
    • changedInput schema / $defs / ClickArgs / properties / count / description
      Previous value: -"Consecutive clicks at this point (default 1, valid range 1 through 10); pass 2 for a\ndouble-click."New value: +"Click count (default 1, range 1 through 10); 2 double-clicks."
    • changedInput schema / $defs / ClickArgs / properties / modifiers / description
      Previous value: -"Modifier keys to hold during the action, e.g. [\"ctrl\"] or [\"ctrl\",\"shift\"] for multi/range-select."New value: +"Held modifiers, e.g. [\"ctrl\", \"shift\"]."
    • changedInput schema / $defs / ClickArgs / properties / x / description
      Previous value: -"Click point x, window-relative — 0 is the window's left edge, not the screen's."New value: +"Click x in window-relative pixels."
    • changedInput schema / $defs / ClickArgs / properties / y / description
      Previous value: -"Click point y, window-relative — 0 is the window's top edge, not the screen's."New value: +"Click y in window-relative pixels."
    • changedInput schema / $defs / ClickElementArgs / properties / id / description
      Previous value: -"Element `#id` from the latest `glass_a11y_snapshot`.\nRe-snapshot after UI changes.\nPopover-owned targets route to their window and restore the prior active window.\nThe role-appropriate native accessibility operation handles occluded or off-screen targets\nand separate labels through their enclosing control.\nUnavailable native operations fall back to a pointer click at the target center.\nText editors may receive focus without activation.\n`method:\"native-action\"` labels any native path.\n`native_fallback` explains pointer fallback.\n`actuated_id` identifies a substituted enclosing control."New value: +"Latest snapshot ID, exclusive with target; re-read after UI changes. Native action may focus text editors; fallback clicks the center. Popover actions restore the prior window."
    • changedInput schema / $defs / ClickElementArgs / properties / id / type
      Previous value: -"integer"New value: +[
      +  "integer",
      +  "null"
      +]
    • addedInput schema / $defs / ClickElementArgs / properties / max_nodes
      Added value: +{
      +  "format": "uint32",
      +  "minimum": 0,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • addedInput schema / $defs / ClickElementArgs / properties / mode
      Added value: +{
      +  "anyOf": [
      +    {
      +      "$ref": "#/$defs/ActionModeArg"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • changedInput schema / $defs / ClickElementArgs / properties / return / description
      Previous value: -"Terminal observation: \"snapshot\" settles and refreshes/folds a11y, \"settle\" waits for\nvisual stability and returns text-only metadata, and \"none\" skips observation (default)."New value: +"none (default): no observation; settle: text-only visual stability; snapshot: settle and refresh/fold the accessibility tree."
    • addedInput schema / $defs / ClickElementArgs / properties / target
      Added value: +{
      +  "anyOf": [
      +    {
      +      "$ref": "#/$defs/ActionTargetArgs"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • addedInput schema / $defs / ClickElementArgs / properties / timeout_ms
      Added value: +{
      +  "format": "uint64",
      +  "maximum": 120000,
      +  "minimum": 0,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • removedInput schema / $defs / ClickElementArgs / required
      Removed value: -[
      -  "id"
      -]
    • changedInput schema / $defs / DiffArgs / properties / ignore / description
      Previous value: -"Window-relative rectangles to exclude from the comparison. Use for\nperpetually animating content — a blinking text caret, a clock, a\nspinner — which otherwise keeps `changed_pct` permanently non-zero.\n`changed_pct` is measured over the pixels that remain; the excluded\ncount is reported as `ignored_pixels`. Combines with `region`: rects are\nalways window-relative and are intersected with it."New value: +"Window-relative excluded rects intersected with region. changed_pct uses remaining pixels; ignored_pixels reports the excluded count."
    • addedInput schema / $defs / DiffArgs / properties / max_height
      Added value: +{
      +  "description": "Maximum returned image height; omit both limits for native output.",
      +  "format": "uint32",
      +  "maximum": 4294967295,
      +  "minimum": 1,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • addedInput schema / $defs / DiffArgs / properties / max_width
      Added value: +{
      +  "description": "Maximum returned image width; shrinks after crop, preserving native comparison pixels.",
      +  "format": "uint32",
      +  "maximum": 4294967295,
      +  "minimum": 1,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • changedInput schema / $defs / DiffArgs / properties / name / description
      Previous value: -"Name of a baseline saved by `glass_baseline_save`; an unsaved name errors\nrather than reporting no change."New value: +"Saved baseline name; unsaved names error."
    • changedInput schema / $defs / DiffArgs / properties / region / description
      Previous value: -"Optional window-relative sub-rectangle to diff; omit to diff the whole\nwindow. Scopes the comparison (and the reported `bbox`, which becomes\nregion-relative) to just this area — the way to ask \"did *only* this part\nchange?\" Mirrors `glass_wait_for_region`'s `region`."New value: +"Window-relative comparison area; omit for whole window. Returned bbox is region-relative."
    • changedInput schema / $defs / DragArgs / properties / duration_ms / description
      Previous value: -"Span the drag's motion over this many milliseconds so a frame-based GUI\nsamples the path across multiple frames (and registers the drag even while\nit repaints). Default 200. Lower = faster but coarser."New value: +"Motion duration in ms (default 200). Faster motion gives the app fewer sampled frames."
    • changedInput schema / $defs / DragArgs / properties / modifiers / description
      Previous value: -"Modifier keys to hold during the action, e.g. [\"ctrl\"] or [\"ctrl\",\"shift\"] for multi/range-select."New value: +"Held modifiers, e.g. [\"ctrl\", \"shift\"]."
    • changedInput schema / $defs / DragArgs / properties / x1 / description
      Previous value: -"Press-point x, window-relative — 0 is the window's left edge, not the screen's."New value: +"Press x in window-relative pixels."
    • changedInput schema / $defs / DragArgs / properties / x2 / description
      Previous value: -"Release-point x, window-relative."New value: +"Release x in window-relative pixels."
    • changedInput schema / $defs / DragArgs / properties / y1 / description
      Previous value: -"Press-point y, window-relative — 0 is the window's top edge, not the screen's."New value: +"Press y in window-relative pixels."
    • changedInput schema / $defs / DragArgs / properties / y2 / description
      Previous value: -"Release-point y, window-relative."New value: +"Release y in window-relative pixels."
    • changedInput schema / $defs / KeyArgs / properties / chord / description
      Previous value: -"A chord like \"ctrl+s\", \"Return\", \"alt+F4\"."New value: +"Modifiers plus one key: ctrl+s, Return, alt+F4. Modifiers: ctrl/shift/alt/super (cmd/win/meta aliases). Key: named key, F1-F12 or printable ASCII; case-insensitive names."
    • changedInput schema / $defs / MoveArgs / properties / x / description
      Previous value: -"Destination x, window-relative — 0 is the window's left edge, not the screen's."New value: +"Destination x in window-relative pixels."
    • changedInput schema / $defs / MoveArgs / properties / y / description
      Previous value: -"Destination y, window-relative — 0 is the window's top edge, not the screen's."New value: +"Destination y in window-relative pixels."
    • changedInput schema / $defs / RegionArgs / properties / height / description
      Previous value: -"Height in pixels, extending down from `y`."New value: +"Height in pixels."
    • changedInput schema / $defs / RegionArgs / properties / width / description
      Previous value: -"Width in pixels, extending right from `x`."New value: +"Width in pixels."
    • changedInput schema / $defs / RegionArgs / properties / x / description
      Previous value: -"Left edge in pixels, window-relative — 0 is the window's left edge, not the screen's."New value: +"Left edge in window-relative pixels."
    • changedInput schema / $defs / RegionArgs / properties / y / description
      Previous value: -"Top edge in pixels, window-relative — 0 is the window's top edge, not the screen's."New value: +"Top edge in window-relative pixels."
    • addedInput schema / $defs / ScreenshotArgs / properties / max_height
      Added value: +{
      +  "description": "Maximum returned image height; omit both limits for native output.",
      +  "format": "uint32",
      +  "maximum": 4294967295,
      +  "minimum": 1,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • addedInput schema / $defs / ScreenshotArgs / properties / max_width
      Added value: +{
      +  "description": "Maximum returned image width; shrinks after crop, preserving native comparison pixels.",
      +  "format": "uint32",
      +  "maximum": 4294967295,
      +  "minimum": 1,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • changedInput schema / $defs / ScreenshotArgs / properties / window_id / description
      Previous value: -"Capture/observe this window (id from `glass_list_windows`) instead of the\nactive one, without changing which window subsequent ops target. Omit for\nthe active window."New value: +"Observe this current glass_list_windows ID without selecting it; omit for active window."
    • changedInput schema / $defs / ScrollArgs / properties / dx / description
      Previous value: -"Horizontal wheel notches from -100 through 100, not pixels.\nPositive is right and negative is left, repeated `|dx|` times.\nTypical values are 1–5."New value: +"Horizontal wheel notches, -100 through 100 (positive right, negative left); typically 1-5, not pixels."
    • changedInput schema / $defs / ScrollArgs / properties / dy / description
      Previous value: -"Vertical wheel notches from -100 through 100, not pixels.\nPositive is down and negative is up, repeated `|dy|` times.\nTypical values are 1–5.\nApps choose how a notch maps to lines, pixels, or zoom."New value: +"Vertical wheel notches, -100 through 100 (positive down, negative up); typically 1-5. App determines distance/zoom."
    • changedInput schema / $defs / ScrollArgs / properties / modifiers / description
      Previous value: -"Modifier keys to hold during the action, e.g. [\"ctrl\"] or [\"ctrl\",\"shift\"] for multi/range-select."New value: +"Held modifiers, e.g. [\"ctrl\", \"shift\"]."
    • changedInput schema / $defs / ScrollArgs / properties / x / description
      Previous value: -"Pointer x the wheel is aimed at, window-relative — apps scroll the container\nunder this point, so it selects which pane moves."New value: +"Window-relative anchor x; selects the container under the pointer."
    • changedInput schema / $defs / ScrollArgs / properties / y / description
      Previous value: -"Pointer y the wheel is aimed at, window-relative. See `x`."New value: +"Window-relative anchor y."
    • changedInput schema / $defs / ScrollToElementArgs / properties / description / description
      Previous value: -"Substring of the target element's accessible description (selector). This can select\nan unnamed control, including an Android text field labelled only by its hint."New value: +"Accessible-description substring; can select unnamed controls."
    • changedInput schema / $defs / ScrollToElementArgs / properties / direction / description
      Previous value: -"Sweep direction: \"up\"/\"down\" (vertical) or \"left\"/\"right\" (horizontal).\nOmit to infer it from the target's off-screen position (falls back to a\nvertical down→up sweep when the target isn't in the a11y tree yet). The\nsearch reverses to the other end if the target isn't found first."New value: +"up/down/left/right. Default infers off-screen direction, or down then up if absent. Reverses at the first end."
    • changedInput schema / $defs / ScrollToElementArgs / properties / step / description
      Previous value: -"Wheel notches per scroll step (default 3). A calibration escape hatch — larger\ncovers distance faster but risks stepping past a row's/column's realized band."New value: +"Wheel notches per step (default 3); large steps can skip realized rows."
    • changedInput schema / $defs / ScrollToElementArgs / properties / timeout_ms / description
      Previous value: -"Give up after this long (default 20000ms); returns `{matched:false}`."New value: +"Timeout in ms (default 20000). Standalone: matched:false; batched: fails the sequence."
    • changedInput schema / $defs / ScrollToElementArgs / properties / value_contains / description
      Previous value: -"Additionally require the matched element's `value` to contain this substring.\nNot a standalone selector — `name`, `description`, and/or `role` is still required."New value: +"Value substring; requires name, description and/or role."
    • changedInput schema / $defs / ScrollToElementArgs / properties / x / description
      Previous value: -"Scroll anchor x (window-relative). By default the swipe anchors on the target's\nown row/column (falling back to the window center if it isn't in the a11y tree\nyet); set both `x` and `y` to point the wheel at a specific container instead."New value: +"Window-relative anchor x; supply both x/y for a container. Default target row/column, or window center if absent."
    • changedInput schema / $defs / SetValueArgs / properties / id / description
      Previous value: -"The element `#id` from `glass_a11y_snapshot`."New value: +"Latest snapshot ID, exclusive with target."
    • changedInput schema / $defs / SetValueArgs / properties / id / type
      Previous value: -"integer"New value: +[
      +  "integer",
      +  "null"
      +]
    • addedInput schema / $defs / SetValueArgs / properties / max_nodes
      Added value: +{
      +  "format": "uint32",
      +  "minimum": 0,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • changedInput schema / $defs / SetValueArgs / properties / return / description
      Previous value: -"Optional observe folded into the result: \"snapshot\" (wait for the UI to settle, then\nfold a fresh a11y tree, also refreshing the snapshot cache), \"settle\" (waits for visual\nstability and returns text-only metadata), or \"none\" (default)."New value: +"none (default): no observation; settle: text-only visual stability; snapshot: settle and refresh/fold the accessibility tree."
    • addedInput schema / $defs / SetValueArgs / properties / target
      Added value: +{
      +  "anyOf": [
      +    {
      +      "$ref": "#/$defs/ActionTargetArgs"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • changedInput schema / $defs / SetValueArgs / properties / text / description
      Previous value: -"The value to set. For a text field, the text. For a spin/slider, a number.\nFor a switch/checkbox/toggle, a boolean (`\"true\"`/`\"false\"`/`\"on\"`/`\"off\"`/\n`\"1\"`/`\"0\"`) — idempotent. For a dropdown/combo box, an option label\n(case-insensitive); glass opens it and picks that option."New value: +"Text, slider/spin number, toggle boolean (true/false/on/off/1/0), or case-insensitive combo option label. Toggle writes are idempotent; combos open and choose the option."
    • addedInput schema / $defs / SetValueArgs / properties / timeout_ms
      Added value: +{
      +  "format": "uint64",
      +  "maximum": 120000,
      +  "minimum": 0,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • changedInput schema / $defs / SetValueArgs / required
      Previous value: -[
      -  "id",
      -  "text"
      -]New value: +[
      +  "text"
      +]
    • changedInput schema / $defs / SettleArgs / properties / ignore / description
      Previous value: -"Window-relative rectangles to exclude from the settle comparison. Use for\nperpetually animating content — a blinking text caret, a clock, a\nspinner — which otherwise keeps the window from ever settling.\nCombines with `stability_region`: rects are always window-relative and\nare intersected with it. A rect that falls partially or entirely\noutside the compared area is silently clamped or dropped, masking less\nthan requested or nothing at all — double-check placement."New value: +"Window-relative excluded rects intersected with stability_region. Off-area rects clamp/drop silently; check placement."
    • changedInput schema / $defs / SettleArgs / properties / stability_region / description
      Previous value: -"Window-relative sub-rectangle to watch for settling; when set, changes outside\nit are ignored."New value: +"Window-relative area watched for settling."
    • changedInput schema / $defs / SettleArgs / properties / timeout_ms / description
      Previous value: -"Give up after this long (default 5000ms).\nThis settle's timeout returns settled:false and completes the step.\nThe enclosing glass_do deadline fails the sequence."New value: +"Timeout in ms (default 5000) completes with settled:false. The overall glass_do deadline instead fails the sequence."
    • changedInput schema / $defs / SettleArgs / properties / tolerance / description
      Previous value: -"Per-channel difference (0–255) two frames may have and still count as\nunchanged (default 0, exact match)."New value: +"Per-channel difference allowed (0-255, default 0)."
    • changedInput schema / $defs / ThenArgs / properties / screenshot / description
      Previous value: -"Capture the window as an image — the only field here that always spends image\ntokens; prefer `diff` when you just need to know whether something changed."New value: +"Return a screenshot. Prefer diff for change detection without image tokens."
    • changedInput schema / $defs / ThenArgs / properties / settle / description
      Previous value: -"Wait for the UI to stop changing first. Set this whenever `diff` or\n`screenshot` follows, or they observe a half-drawn frame."New value: +"Wait for quiescence before diff/screenshot; otherwise they may observe a half-drawn frame."
    • addedInput schema / $defs / TypeArgs / properties / focus_mode
      Added value: +{
      +  "anyOf": [
      +    {
      +      "$ref": "#/$defs/ActionModeArg"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • addedInput schema / $defs / TypeArgs / properties / max_nodes
      Added value: +{
      +  "format": "uint32",
      +  "minimum": 0,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • changedInput schema / $defs / TypeArgs / properties / return / description
      Previous value: -"Terminal observation: \"snapshot\" settles and refreshes/folds a11y, \"settle\" waits for\nvisual stability and returns text-only metadata, and \"none\" skips observation (default)."New value: +"none (default): no observation; settle: text-only visual stability; snapshot: settle and read the current accessibility tree, including app-visible text, with secure values redacted."
    • addedInput schema / $defs / TypeArgs / properties / target
      Added value: +{
      +  "anyOf": [
      +    {
      +      "$ref": "#/$defs/ActionTargetArgs"
      +    },
      +    {
      +      "type": "null"
      +    }
      +  ]
      +}
    • changedInput schema / $defs / TypeArgs / properties / text / description
      Previous value: -"Text to type into whatever currently has keyboard focus — this tool does not\nfocus a field, so click or `glass_click_element` one first. Sent as synthetic\nkey events, not pasted, so an app's per-keystroke handlers run."New value: +"Synthetic keystrokes, not paste. When `target` is omitted: current keyboard focus. When `target` is supplied: resolves and focuses the target, confirms focus, then types. Newlines do not press Return."
    • addedInput schema / $defs / TypeArgs / properties / timeout_ms
      Added value: +{
      +  "format": "uint64",
      +  "maximum": 120000,
      +  "minimum": 0,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
    • changedInput schema / $defs / WaitForElementArgs / properties / condition / description
      Previous value: -"What to wait for (default \"appears\"): appears|disappears|enabled|disabled|\nchecked|unchecked|selected|unselected|expanded|collapsed|focused|visible|hidden.\n`checked`/`unchecked` only match a checkable element (one exposing a real toggle\nstate) — a non-toggle element matches neither."New value: +"Default appears. appears|disappears|enabled|disabled|checked|unchecked|selected|unselected|expanded|collapsed|focused|visible|hidden. checked/unchecked require a real checkable state."
    • changedInput schema / $defs / WaitForElementArgs / properties / description / description
      Previous value: -"Substring of the element's accessible description (selector). Useful for unnamed\ncontrols whose platform label is exposed as a hint, help text, or description."New value: +"Accessible-description substring; can select unnamed controls."
    • changedInput schema / $defs / WaitForElementArgs / properties / timeout_ms / description
      Previous value: -"Give up after this long (default 10000ms); returns `{matched:false}`."New value: +"Timeout in ms (default 10000). Standalone: matched:false; batched: fails the sequence."
    • changedInput schema / $defs / WaitForElementArgs / properties / value_contains / description
      Previous value: -"Additionally require the matched element's `value` to contain this substring.\nNot a standalone selector — `name`, `description`, and/or `role` is still required."New value: +"Value substring; requires name, description and/or role; exclusive with value."
    • changedInput schema / properties / actions / description
      Previous value: -"Actions to run in order; must be non-empty. Fail-fast — the first failing\naction aborts the rest and reports its index, so a partial sequence may\nalready have landed."New value: +"Non-empty ordered actions; failure stops remaining steps. Completed mutations may already have landed."
    • changedInput schema / properties / then / description
      Previous value: -"Optional observe run once after the last action, in the order settle → diff\n→ screenshot."New value: +"Terminal observation in order: settle, diff, screenshot."
    • changedInput schema / properties / timeout_ms / description
      Previous value: -"Overall sequence budget in milliseconds. Omit for 30000; valid range\n1..=120000. One absolute deadline is shared by all actions and terminal\nobservations."New value: +"Overall budget in ms (default 30000, range 1..120000), shared by all actions and terminal observations."
  2. Changed17 schema fields changedv1.5.1
    • changedInput schema / $defs / Action / oneOf
      Previous value: -[
      -  {
      -    "$ref": "#/$defs/ClickArgs",
      -    "properties": {
      -      "action": {
      -        "const": "click",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "action"
      -    ],
      -    "type": "object"
      -  },
      -  {
      -    "$ref": "#/$defs/MoveArgs",
      -    "properties": {
      -      "action": {
      -        "const": "move",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "action"
      -    ],
      -    "type": "object"
      -  },
      -  {
      -    "$ref": "#/$defs/DragArgs",
      -    "properties": {
      -      "action": {
      -        "const": "drag",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "action"
      -    ],
      -    "type": "object"
      -  },
      -  {
      -    "$ref": "#/$defs/ScrollArgs",
      -    "properties": {
      -      "action": {
      -        "const": "scroll",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "action"
      -    ],
      -    "type": "object"
      -  },
      -  {
      -    "$ref": "#/$defs/TypeArgs",
      -    "properties": {
      -      "action": {
      -        "const": "type",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "action"
      -    ],
      -    "type": "object"
      -  },
      -  {
      -    "$ref": "#/$defs/KeyArgs",
      -    "properties": {
      -      "action": {
      -        "const": "key",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "action"
      -    ],
      -    "type": "object"
      -  },
      -  {
      -    "$ref": "#/$defs/SettleArgs",
      -    "properties": {
      -      "action": {
      -        "const": "settle",
      -        "type": "string"
      -      }
      -    },
      -    "required": [
      -      "action"
      -    ],
      -    "type": "object"
      -  }
      -]New value: +[
      +  {
      +    "$ref": "#/$defs/ClickArgs",
      +    "properties": {
      +      "action": {
      +        "const": "click",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/MoveArgs",
      +    "properties": {
      +      "action": {
      +        "const": "move",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/DragArgs",
      +    "properties": {
      +      "action": {
      +        "const": "drag",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/ScrollArgs",
      +    "properties": {
      +      "action": {
      +        "const": "scroll",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/TypeArgs",
      +    "properties": {
      +      "action": {
      +        "const": "type",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/KeyArgs",
      +    "properties": {
      +      "action": {
      +        "const": "key",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/SettleArgs",
      +    "properties": {
      +      "action": {
      +        "const": "settle",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/ClickElementArgs",
      +    "properties": {
      +      "action": {
      +        "const": "click_element",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/SetValueArgs",
      +    "properties": {
      +      "action": {
      +        "const": "set_value",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/WaitForElementArgs",
      +    "properties": {
      +      "action": {
      +        "const": "wait_for_element",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  },
      +  {
      +    "$ref": "#/$defs/ScrollToElementArgs",
      +    "properties": {
      +      "action": {
      +        "const": "scroll_to_element",
      +        "type": "string"
      +      }
      +    },
      +    "required": [
      +      "action"
      +    ],
      +    "type": "object"
      +  }
      +]
    • changedInput schema / $defs / ClickArgs / properties / count / description
      Previous value: -"Consecutive clicks at this point (default 1); pass 2 for a double-click."New value: +"Consecutive clicks at this point (default 1, valid range 1 through 10); pass 2 for a\ndouble-click."
    • addedInput schema / $defs / ClickArgs / properties / count / maximum
      Added value: +10
    • changedInput schema / $defs / ClickArgs / properties / count / minimum
      Previous value: -0New value: +1
    • addedInput schema / $defs / ClickElementArgs
      Added value: +{
      +  "properties": {
      +    "id": {
      +      "description": "Element `#id` from the latest `glass_a11y_snapshot`.\nRe-snapshot after UI changes.\nPopover-owned targets route to their window and restore the prior active window.\nThe role-appropriate native accessibility operation handles occluded or off-screen targets\nand separate labels through their enclosing control.\nUnavailable native operations fall back to a pointer click at the target center.\nText editors may receive focus without activation.\n`method:\"native-action\"` labels any native path.\n`native_fallback` explains pointer fallback.\n`actuated_id` identifies a substituted enclosing control.",
      +      "format": "uint32",
      +      "minimum": 0,
      +      "type": "integer"
      +    },
      +    "return": {
      +      "description": "Terminal observation: \"snapshot\" settles and refreshes/folds a11y, \"settle\" waits for\nvisual stability and returns text-only metadata, and \"none\" skips observation (default).",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    }
      +  },
      +  "required": [
      +    "id"
      +  ],
      +  "type": "object"
      +}
    • changedInput schema / $defs / ScrollArgs / properties / dx / description
      Previous value: -"Horizontal scroll in **wheel notches** (discrete clicks — small integers like 1–5, NOT\npixels). Positive `dx` sends wheel-right, negative wheel-left; glass clicks `|dx|` times."New value: +"Horizontal wheel notches from -100 through 100, not pixels.\nPositive is right and negative is left, repeated `|dx|` times.\nTypical values are 1–5."
    • addedInput schema / $defs / ScrollArgs / properties / dx / maximum
      Added value: +100
    • addedInput schema / $defs / ScrollArgs / properties / dx / minimum
      Added value: +-100
    • changedInput schema / $defs / ScrollArgs / properties / dy / description
      Previous value: -"Vertical scroll in **wheel notches** (discrete clicks — small integers like 1–5, NOT\npixels). Positive `dy` sends wheel-down, negative wheel-up; glass clicks `|dy|` times. How\nan app maps a wheel notch to its view (lines, pixels, zoom) is the app's choice."New value: +"Vertical wheel notches from -100 through 100, not pixels.\nPositive is down and negative is up, repeated `|dy|` times.\nTypical values are 1–5.\nApps choose how a notch maps to lines, pixels, or zoom."
    • addedInput schema / $defs / ScrollArgs / properties / dy / maximum
      Added value: +100
    • addedInput schema / $defs / ScrollArgs / properties / dy / minimum
      Added value: +-100
    • addedInput schema / $defs / ScrollToElementArgs
      Added value: +{
      +  "properties": {
      +    "description": {
      +      "description": "Substring of the target element's accessible description (selector). This can select\nan unnamed control, including an Android text field labelled only by its hint.",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "direction": {
      +      "description": "Sweep direction: \"up\"/\"down\" (vertical) or \"left\"/\"right\" (horizontal).\nOmit to infer it from the target's off-screen position (falls back to a\nvertical down→up sweep when the target isn't in the a11y tree yet). The\nsearch reverses to the other end if the target isn't found first.",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "name": {
      +      "description": "Substring of the target element's accessible name (selector).",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "role": {
      +      "description": "Element role filter, e.g. \"ListItem\", \"Button\", \"Document\" (selector).",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "step": {
      +      "description": "Wheel notches per scroll step (default 3). A calibration escape hatch — larger\ncovers distance faster but risks stepping past a row's/column's realized band.",
      +      "format": "uint32",
      +      "minimum": 0,
      +      "type": [
      +        "integer",
      +        "null"
      +      ]
      +    },
      +    "timeout_ms": {
      +      "description": "Give up after this long (default 20000ms); returns `{matched:false}`.",
      +      "format": "uint64",
      +      "minimum": 0,
      +      "type": [
      +        "integer",
      +        "null"
      +      ]
      +    },
      +    "value_contains": {
      +      "description": "Additionally require the matched element's `value` to contain this substring.\nNot a standalone selector — `name`, `description`, and/or `role` is still required.",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "x": {
      +      "description": "Scroll anchor x (window-relative). By default the swipe anchors on the target's\nown row/column (falling back to the window center if it isn't in the a11y tree\nyet); set both `x` and `y` to point the wheel at a specific container instead.",
      +      "format": "int32",
      +      "type": [
      +        "integer",
      +        "null"
      +      ]
      +    },
      +    "y": {
      +      "description": "Scroll anchor y (window-relative). See `x`.",
      +      "format": "int32",
      +      "type": [
      +        "integer",
      +        "null"
      +      ]
      +    }
      +  },
      +  "type": "object"
      +}
    • addedInput schema / $defs / SetValueArgs
      Added value: +{
      +  "properties": {
      +    "id": {
      +      "description": "The element `#id` from `glass_a11y_snapshot`.",
      +      "format": "uint32",
      +      "minimum": 0,
      +      "type": "integer"
      +    },
      +    "return": {
      +      "description": "Optional observe folded into the result: \"snapshot\" (wait for the UI to settle, then\nfold a fresh a11y tree, also refreshing the snapshot cache), \"settle\" (waits for visual\nstability and returns text-only metadata), or \"none\" (default).",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "text": {
      +      "description": "The value to set. For a text field, the text. For a spin/slider, a number.\nFor a switch/checkbox/toggle, a boolean (`\"true\"`/`\"false\"`/`\"on\"`/`\"off\"`/\n`\"1\"`/`\"0\"`) — idempotent. For a dropdown/combo box, an option label\n(case-insensitive); glass opens it and picks that option.",
      +      "type": "string"
      +    }
      +  },
      +  "required": [
      +    "id",
      +    "text"
      +  ],
      +  "type": "object"
      +}
    • changedInput schema / $defs / SettleArgs / properties / timeout_ms / description
      Previous value: -"Give up after this long (default 5000ms); the sequence continues rather than\nfailing."New value: +"Give up after this long (default 5000ms).\nThis settle's timeout returns settled:false and completes the step.\nThe enclosing glass_do deadline fails the sequence."
    • changedInput schema / $defs / TypeArgs / properties / return / description
      Previous value: -"Optional observe folded into the result: \"snapshot\" (wait for the UI to settle, then\nfold a fresh a11y tree, also refreshing the snapshot cache), \"settle\" (wait for the UI\nto stop changing, text-only), or \"none\" (default). Not accepted inside a `glass_do`\n`type` action — use a `settle` action or the terminal `then` observe there."New value: +"Terminal observation: \"snapshot\" settles and refreshes/folds a11y, \"settle\" waits for\nvisual stability and returns text-only metadata, and \"none\" skips observation (default)."
    • addedInput schema / $defs / WaitForElementArgs
      Added value: +{
      +  "properties": {
      +    "condition": {
      +      "description": "What to wait for (default \"appears\"): appears|disappears|enabled|disabled|\nchecked|unchecked|selected|unselected|expanded|collapsed|focused|visible|hidden.\n`checked`/`unchecked` only match a checkable element (one exposing a real toggle\nstate) — a non-toggle element matches neither.",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "description": {
      +      "description": "Substring of the element's accessible description (selector). Useful for unnamed\ncontrols whose platform label is exposed as a hint, help text, or description.",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "interval_ms": {
      +      "description": "Poll interval (default 200ms — an a11y snapshot per tick).",
      +      "format": "uint64",
      +      "minimum": 0,
      +      "type": [
      +        "integer",
      +        "null"
      +      ]
      +    },
      +    "name": {
      +      "description": "Substring of the element's accessible name (selector).",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "role": {
      +      "description": "Element role filter, e.g. \"Button\", \"ProgressBar\", \"Document\" (selector).",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "timeout_ms": {
      +      "description": "Give up after this long (default 10000ms); returns `{matched:false}`.",
      +      "format": "uint64",
      +      "minimum": 0,
      +      "type": [
      +        "integer",
      +        "null"
      +      ]
      +    },
      +    "value": {
      +      "description": "Exact case-sensitive accessible value, requiring another selector and excluding `value_contains`.",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    },
      +    "value_contains": {
      +      "description": "Additionally require the matched element's `value` to contain this substring.\nNot a standalone selector — `name`, `description`, and/or `role` is still required.",
      +      "type": [
      +        "string",
      +        "null"
      +      ]
      +    }
      +  },
      +  "type": "object"
      +}
    • addedInput schema / properties / timeout_ms
      Added value: +{
      +  "description": "Overall sequence budget in milliseconds. Omit for 30000; valid range\n1..=120000. One absolute deadline is shared by all actions and terminal\nobservations.",
      +  "format": "uint64",
      +  "minimum": 0,
      +  "type": [
      +    "integer",
      +    "null"
      +  ]
      +}
  3. Changed31 schema fields changedv1.2.0
    • addedInput schema / $defs / ClickArgs / properties / count / description
      Added value: +"Consecutive clicks at this point (default 1); pass 2 for a double-click."
    • addedInput schema / $defs / ClickArgs / properties / x / description
      Added value: +"Click point x, window-relative — 0 is the window's left edge, not the screen's."
    • addedInput schema / $defs / ClickArgs / properties / y / description
      Added value: +"Click point y, window-relative — 0 is the window's top edge, not the screen's."
    • addedInput schema / $defs / DiffArgs / properties / name / description
      Added value: +"Name of a baseline saved by `glass_baseline_save`; an unsaved name errors\nrather than reporting no change."
    • addedInput schema / $defs / DragArgs / properties / button / description
      Added value: +"Button held for the drag: \"left\" (default), \"right\", or \"middle\"."
    • addedInput schema / $defs / DragArgs / properties / x1 / description
      Added value: +"Press-point x, window-relative — 0 is the window's left edge, not the screen's."
    • addedInput schema / $defs / DragArgs / properties / x2 / description
      Added value: +"Release-point x, window-relative."
    • addedInput schema / $defs / DragArgs / properties / y1 / description
      Added value: +"Press-point y, window-relative — 0 is the window's top edge, not the screen's."
    • addedInput schema / $defs / DragArgs / properties / y2 / description
      Added value: +"Release-point y, window-relative."
    • addedInput schema / $defs / MoveArgs / properties / x / description
      Added value: +"Destination x, window-relative — 0 is the window's left edge, not the screen's."
    • addedInput schema / $defs / MoveArgs / properties / y / description
      Added value: +"Destination y, window-relative — 0 is the window's top edge, not the screen's."
    • addedInput schema / $defs / RegionArgs / properties / height / description
      Added value: +"Height in pixels, extending down from `y`."
    • addedInput schema / $defs / RegionArgs / properties / width / description
      Added value: +"Width in pixels, extending right from `x`."
    • addedInput schema / $defs / RegionArgs / properties / x / description
      Added value: +"Left edge in pixels, window-relative — 0 is the window's left edge, not the screen's."
    • addedInput schema / $defs / RegionArgs / properties / y / description
      Added value: +"Top edge in pixels, window-relative — 0 is the window's top edge, not the screen's."
    • addedInput schema / $defs / ScrollArgs / properties / x / description
      Added value: +"Pointer x the wheel is aimed at, window-relative — apps scroll the container\nunder this point, so it selects which pane moves."
    • addedInput schema / $defs / ScrollArgs / properties / y / description
      Added value: +"Pointer y the wheel is aimed at, window-relative. See `x`."
    • addedInput schema / $defs / SettleArgs / properties / interval_ms / description
      Added value: +"How long to wait between capture ticks (default 100ms)."
    • addedInput schema / $defs / SettleArgs / properties / settle_frames / description
      Added value: +"Consecutive unchanged frames required before the UI counts as settled (default 3)."
    • addedInput schema / $defs / SettleArgs / properties / stability_region / description
      Added value: +"Window-relative sub-rectangle to watch for settling; when set, changes outside\nit are ignored."
    • addedInput schema / $defs / SettleArgs / properties / timeout_ms / description
      Added value: +"Give up after this long (default 5000ms); the sequence continues rather than\nfailing."
    • addedInput schema / $defs / SettleArgs / properties / tolerance / description
      Added value: +"Per-channel difference (0–255) two frames may have and still count as\nunchanged (default 0, exact match)."
    • addedInput schema / $defs / ThenArgs / properties / diff / description
      Added value: +"Compare against a saved baseline and return change stats as text."
    • addedInput schema / $defs / ThenArgs / properties / screenshot / description
      Added value: +"Capture the window as an image — the only field here that always spends image\ntokens; prefer `diff` when you just need to know whether something changed."
    • addedInput schema / $defs / ThenArgs / properties / settle / description
      Added value: +"Wait for the UI to stop changing first. Set this whenever `diff` or\n`screenshot` follows, or they observe a half-drawn frame."
    • addedInput schema / $defs / TypeArgs / properties / return
      Added value: +{
      +  "description": "Optional observe folded into the result: \"snapshot\" (wait for the UI to settle, then\nfold a fresh a11y tree, also refreshing the snapshot cache), \"settle\" (wait for the UI\nto stop changing, text-only), or \"none\" (default). Not accepted inside a `glass_do`\n`type` action — use a `settle` action or the terminal `then` observe there.",
      +  "type": [
      +    "string",
      +    "null"
      +  ]
      +}
    • addedInput schema / $defs / TypeArgs / properties / text / description
      Added value: +"Text to type into whatever currently has keyboard focus — this tool does not\nfocus a field, so click or `glass_click_element` one first. Sent as synthetic\nkey events, not pasted, so an app's per-keystroke handlers run."
    • removedInput schema / description
      Removed value: -"Arguments for `glass_do`: an ordered, non-empty action sequence + optional observe."
    • addedInput schema / properties / actions / description
      Added value: +"Actions to run in order; must be non-empty. Fail-fast — the first failing\naction aborts the rest and reports its index, so a partial sequence may\nalready have landed."
    • addedInput schema / properties / then / description
      Added value: +"Optional observe run once after the last action, in the order settle → diff\n→ screenshot."
    • removedInput schema / title
      Removed value: -"DoArgs"
  4. Addedv1.1.0
  5. Removedv1.0.3
  6. Addedv1.0.2

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description goes far beyond the annotations, which only declare readOnlyHint=false, destructiveHint=false, openWorldHint=false. It discloses fail-fast behavior, the shared deadline semantics, default timeout and range, what 'ok:true means', the structure of success vs. failure returns, terminal settle/diff/screenshot behavior, preflight invalid_sequence behavior, semantic target freshness, and a critical warning: 'Never replay completed or possibly dispatched mutations; observe after uncertainty.' This is precisely the kind of behavioral context an agent needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense and packs a lot of information, but every sentence contributes a distinct fact: limits, defaults, failure modes, terminal observation, and restrictions. It is front-loaded with the core purpose and the accepted action list. It is not padded, though it is a long wall of text for a tool definition; a slightly more structured layout (e.g., headings) could improve scanability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex batching tool with no output schema, the description covers all the essentials an agent needs to call it correctly: the action vocabulary, size limits, timeout behavior, success/failure return structures, terminal observation semantics, and critical caveats about mutations and semantic targets. It even explains what happens on preflight invalid_sequence. Nothing critical is missing given the schema already documents individual action arguments.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although the schema covers 100% of parameters with descriptions, the tool description adds operational meaning not present in the schema: 'Maximum 64 actions and 65536 compact argument bytes,' the default timeout (30000ms) and range, the single shared deadline for all actions and observations, and the return contract ('ordered steps with result and content_blocks'). It also clarifies the semantics of `then` ordering (settle → diff → screenshot) and how failures retain `completed, failed, unexecuted` steps. This is substantial additional meaning, not mere repetition.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb and resource: 'Batch 2+ known actions or waits here; observe before choosing dependent steps.' It distinguishes itself from the 30 sibling tools by positioning glass_do as the batching/sequencing tool, and enumerates the exact action types it accepts (click, move, drag, scroll, type, key, settle, etc.), leaving no ambiguity about what it does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use it: when you have multiple known actions or waits that should run as an ordered sequence, with observation between batches. It also states what it cannot do ('No branching, bindings, loops, retries or generated steps'), which helps an agent avoid misusing it. It doesn't explicitly say 'use this instead of calling standalone sibling tools individually,' but the context of batching makes that evident.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.