Skip to main content
Glama

mouse_drag

Click and drag with the left mouse button from a start to an end coordinate to move sliders, drop items, draw on canvases, or resize windows, with guards that block accidental cross-window drags.

Instructions

Click and drag from (startX, startY) to (endX, endY) holding the left mouse button — for sliders, drag-and-drop, canvas drawing, and window resizing. Pass windowTitle so the server auto-guards the start coordinate and returns post.perception. Examples: mouse_drag({windowTitle:'Notepad', startX:50, startY:50, endX:200, endY:200}). lensId is optional and only for advanced pinned-target workflows. Caveats: Left button only. Both start and endpoint are guarded. Cross-window and desktop drags are blocked by default — pass allowCrossWindowDrag:true to confirm intent; that refusal is code:'CrossWindowDragBlocked'. A drag starting in a tabbed application's tab strip returns code:'TabDragBlocked' — pass allowTabDrag:true when tearing off or rearranging a tab is intended. hints.verifyDelivery:{status:'delivered'|'focus_only'|'unverifiable', reason} reports the post-drop observation in the same 3-value shape as mouse_click. MouseDragNotDelivered is SUGGESTS-registered but reserved-only (not emitted) — degradation is expressed via the 'unverifiable' status rather than a typed code. Win11 foreground refusal (UIPI cross-elevation / admin-only target / call from a background process or service) returns code:'ForegroundRestricted' ok:false from the homing path. A window on another virtual desktop returns WindowOnOtherDesktop and nothing is sent — the user's desktop is not switched.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
endXYes
endYYes
hwndNoDirect window handle ID (takes precedence over windowTitle). Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues.
speedNoCursor movement speed in px/sec. 0 = instant (teleport, no animation). Omit to use the configured default (DESKTOP_TOUCH_MOUSE_SPEED env var, default 3000).
homingNoEnable homing correction (default true). When enabled, the MCP server corrects stale coordinates if the target window moved since the last screenshot. Set false to disable all correction (like traction control OFF).
lensIdNoOptional perception lens ID. Guards and envelope same as mouse_click.
startXYes
startYYes
includeNoOptional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients).
narrateNoNarration level. "rich": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — "@active" and the dialog rescue both resolve one ("ambiguous_title"); or the target moved between the snapshot and the action ("target_changed"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see ("fix_target_unknown") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too ("tree_truncated"). The action itself is unaffected in every case; only the diff is. Default: "minimal".minimal
windowTitleNoHint: partial title of the window being clicked. Enables window-delta correction and auto-focus if the window went behind another. Use '@active' for the current foreground window. Example: "メモ帳", "Google Chrome"
allowTabDragNoWhen true, allow drags that start in the title-bar / tab-strip area of a tabbed app (Notepad, Terminal, Edge, Chrome, etc.). Default false — such drags are blocked because they detach the tab into a new window rather than moving the window. Pass true only when you intentionally want to rearrange or detach a tab. Note: active only when auto-guard is enabled (same scope as allowCrossWindowDrag).
verifyDeliveryNoWhen true (default), capture pre/post snapshots of element-under-cursor + focusedElement + foregroundWindow + scrollPos to populate hints.verifyDelivery with status='delivered' | 'focus_only' | 'unverifiable' (issue #178). Set false to skip the extra UIA work when the caller will read post state itself.
allowCrossWindowDragNoWhen true, allow dragging the endpoint into a different window or the desktop background. Default false — cross-window drags (including desktop/wallpaper) are blocked to prevent accidents. Pass true to confirm intent for deliberate cross-window or desktop-area drags.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed12 schema fields changedv2.3.1
    • addedInput schema / $schema
      Added value: +"http://json-schema.org/draft-07/schema#"
    • removedInput schema / additionalProperties
      Removed value: -false
    • changedInput schema / properties / homing / description
      Previous value: -"Enable homing correction if the target window moved."New value: +"Enable homing correction (default true). When enabled, the MCP server corrects stale coordinates if the target window moved since the last screenshot. Set false to disable all correction (like traction control OFF)."
    • changedInput schema / properties / hwnd / description
      Previous value: -"Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues."New value: +"Direct window handle ID (takes precedence over windowTitle). Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues."
    • changedInput schema / properties / narrate / description
      Previous value: -"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."New value: +"Narration level. \"rich\": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — \"@active\" and the dialog rescue both resolve one (\"ambiguous_title\"); or the target moved between the snapshot and the action (\"target_changed\"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see (\"fix_target_unknown\") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too (\"tree_truncated\"). The action itself is unaffected in every case; only the diff is. Default: \"minimal\"."
    • changedInput schema / properties / speed / description
      Previous value: -"Cursor movement speed in px/sec. 0 = instant."New value: +"Cursor movement speed in px/sec. 0 = instant (teleport, no animation). Omit to use the configured default (DESKTOP_TOUCH_MOUSE_SPEED env var, default 3000)."
    • addedInput schema / properties / speed / maximum
      Added value: +9007199254740991
    • addedInput schema / properties / verifyDelivery / default
      Added value: +true
    • changedInput schema / properties / verifyDelivery / description
      Previous value: -"Parameter 'verifyDeliveryParam' from the Windows server schema."New value: +"When true (default), capture pre/post snapshots of element-under-cursor + focusedElement + foregroundWindow + scrollPos to populate hints.verifyDelivery with status='delivered' | 'focus_only' | 'unverifiable' (issue #178). Set false to skip the extra UIA work when the caller will read post state itself."
    • addedInput schema / properties / verifyDelivery / type
      Added value: +"boolean"
    • changedInput schema / properties / windowTitle / description
      Previous value: -"Partial title of the target window."New value: +"Hint: partial title of the window being clicked. Enables window-delta correction and auto-focus if the window went behind another. Use '@active' for the current foreground window. Example: \"メモ帳\", \"Google Chrome\""
    • changedInput schema / required
      Previous value: -[
      -  "startX",
      -  "startY",
      -  "endX",
      -  "endY",
      -  "verifyDelivery"
      -]New value: +[
      +  "startX",
      +  "startY",
      +  "endX",
      +  "endY"
      +]
  2. Changed1 schema field changedv2.0.0
    • changedInput schema / properties / narrate / description
      Previous value: -"Narration level. rich includes UIA or browser state diff when supported."New value: +"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."
  3. Addedv1.9.2
  4. Removedv1.8.0
  5. Addedv1.6.0
  6. Removedv1.5.1
  7. Addedv1.5.0
  8. Removedv1.4.3
  9. Addedv1.4.2
  10. Removedv1.4.0
  11. Changed1 schema field changedv1.2.1
    • addedInput schema / properties / include
      Added value: +{
      +  "description": "Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients).",
      +  "items": {
      +    "type": "string"
      +  },
      +  "type": "array"
      +}
  12. Addedv1.0.3
  13. Removedv0.15.7
  14. Changed1 schema field changedv0.15.4
    • addedInput schema / properties / hwnd
      Added value: +{
      +  "description": "Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues.",
      +  "type": "string"
      +}
  15. Changed1 schema field changedv0.15.3
    • addedInput schema / properties / allowTabDrag
      Added value: +{
      +  "default": false,
      +  "description": "When true, allow drags that start in the title-bar / tab-strip area of a tabbed app (Notepad, Terminal, Edge, Chrome, etc.). Default false — such drags are blocked because they detach the tab into a new window rather than moving the window. Pass true only when you intentionally want to rearrange or detach a tab. Note: active only when auto-guard is enabled (same scope as allowCrossWindowDrag).",
      +  "type": "boolean"
      +}
  16. Changed1 schema field changedv0.13.1
    • addedInput schema / properties / allowCrossWindowDrag
      Added value: +{
      +  "default": false,
      +  "description": "When true, allow dragging the endpoint into a different window or the desktop background. Default false — cross-window drags (including desktop/wallpaper) are blocked to prevent accidents. Pass true to confirm intent for deliberate cross-window or desktop-area drags.",
      +  "type": "boolean"
      +}
  17. Changed13 schema fields changedv0.10.0
    • removedInput schema / $schema
      Removed value: -"http://json-schema.org/draft-07/schema#"
    • addedInput schema / additionalProperties
      Added value: +false
    • removedInput schema / properties / _stub
      Removed value: -{
      -  "description": "Stub tool — arguments ignored."
      -}
    • addedInput schema / properties / endX
      Added value: +{
      +  "type": "number"
      +}
    • addedInput schema / properties / endY
      Added value: +{
      +  "type": "number"
      +}
    • addedInput schema / properties / homing
      Added value: +{
      +  "default": true,
      +  "description": "Enable homing correction if the target window moved.",
      +  "type": "boolean"
      +}
    • addedInput schema / properties / lensId
      Added value: +{
      +  "description": "Optional perception lens ID. Guards and envelope same as mouse_click.",
      +  "type": "string"
      +}
    • addedInput schema / properties / narrate
      Added value: +{
      +  "default": "minimal",
      +  "description": "Narration level. rich includes UIA or browser state diff when supported.",
      +  "enum": [
      +    "minimal",
      +    "rich"
      +  ],
      +  "type": "string"
      +}
    • addedInput schema / properties / speed
      Added value: +{
      +  "description": "Cursor movement speed in px/sec. 0 = instant.",
      +  "minimum": 0,
      +  "type": "integer"
      +}
    • addedInput schema / properties / startX
      Added value: +{
      +  "type": "number"
      +}
    • addedInput schema / properties / startY
      Added value: +{
      +  "type": "number"
      +}
    • addedInput schema / properties / windowTitle
      Added value: +{
      +  "description": "Partial title of the target window.",
      +  "type": "string"
      +}
    • addedInput schema / required
      Added value: +[
      +  "startX",
      +  "startY",
      +  "endX",
      +  "endY"
      +]
  18. First observedv0.6.2

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so thoroughly: left-button-only, both endpoints guarded, cross-window/desktop blocked by default, tab-strip drags blocked, plus specific error codes (CrossWindowDragBlocked, TabDragBlocked, ForegroundRestricted, WindowOnOtherDesktop) and the verifyDelivery status semantics. It also discloses that a window on another virtual desktop sends nothing rather than switching desktops.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action, endpoint semantics, and use cases before diving into caveats, which is the right order. It is dense but most sentences earn their place; the niche 'MouseDragNotDelivered is SUGGESTS-registered but reserved-only' aside is the main place where trimming would help.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description must carry return semantics, and it covers them: post.perception, hints.verifyDelivery statuses, and the error-code taxonomy for blocked or refused drags. It is nearly complete for a 14-parameter mutation tool, with only minor gaps around the untouched params.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 71%, and the description adds genuine meaning beyond the schema for several parameters: windowTitle 'auto-guards the start coordinate and returns post.perception', lensId is scoped as advanced-only, and the two guard flags are framed by intent. It leaves some params (speed, homing, include, narrate, hwnd) entirely to the schema, so it does not fully compensate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource with both endpoints ('Click and drag from (startX, startY) to (endX, endY)') and enumerates concrete use cases (sliders, drag-and-drop, canvas drawing, window resizing). It implicitly and explicitly distinguishes itself from the sibling mouse_click by referencing the shared verifyDelivery shape and by its distinct drag semantics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear when-to-use context via the use-case list and a worked example, plus explicit conditions for the advanced flags (pass allowCrossWindowDrag:true to confirm cross-window intent; allowTabDrag:true for tearing off tabs). It does not, however, explicitly route the agent to mouse_click or scroll as alternatives, so it stops short of full when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.