desktop-touch-mcp
desktop-touch-mcp is a Windows computer-use MCP server that lets an AI observe and operate the desktop and Chrome/Edge — 32 tools covering semantic UI automation, browser CDP, keyboard/mouse, terminal, and Excel VBA, with per-action perception guards that refuse impossible actions instead of guessing.
Observe state —
desktop_state(focused window/element, modal, cursor, optional screen/document/session info),screenshot(meta/text/image/som/ocrdetail,diffModeP-frames,dotByDot1:1 coords,backgroundwindow capture),workspace_snapshot,screenshot_query/screenshot_gcfor the on-disk capture cache,server_statusdiagnostics.Act on native UI —
desktop_discover/desktop_act(lease-based discover-then-act with semantic diffs androiCapture),click_element(UIA InvokePattern),keyboard(type/press/sequence, IME-safe, background WM_CHAR),mouse_click/mouse_drag,scroll(raw/to_element/smart/capture/read),focus_window,window_dock(pin/dock),wait_untilpolling.Drive browsers (Chrome/Edge/Brave CDP) —
browser_open/browser_navigate,browser_click/browser_fill/browser_form(selector or semantic by-axis targeting),browser_eval(js/dom/appStateSPA state extraction),browser_overview/browser_search/browser_locate.Run terminals —
terminalrun/read/sendwith quiet/pattern/exit completion, OCR or UIA read-back, pluskey_lockerfor encrypted SSH/sudo credential autofill in a locker-launched console.Automate Excel —
excelauthors and runs VBA macros via COM (orcheck_access_vbomread-only preflight) where formula-only tools cannot.Batch and exchange data —
run_macro(up to 50 steps in one round trip),clipboardread/write with byte-verified delivery,notification_show,workspace_launchto start apps and detect their windows.Safety and reliability throughout — discover-then-act targeting, per-action guards (identity, foreground, modal, virtual desktop) that refuse undeliverable input with a reason, delivery verification hints, CJK/IME support, Rust native core with PowerShell fallback.
Provides browser automation capabilities through CDP (Chrome DevTools Protocol), enabling connection to Brave browser, finding elements via CSS selectors, clicking elements, evaluating JavaScript, and navigating pages without requiring Selenium or Playwright.
Supports CSS selector-based element location in browser automation, allowing precise identification of DOM elements in Chrome/Edge/Brave browsers for clicking and interaction through CDP integration.
Provides comprehensive browser automation through CDP (Chrome DevTools Protocol), enabling connection to Chrome, finding elements via CSS selectors, clicking elements, evaluating JavaScript, getting DOM content, and navigating pages without requiring Selenium or Playwright.
desktop-touch-mcp
Computer-use MCP server for Windows. Lets Claude, Cursor, or any MCP client see and operate your Windows 10/11 desktop — screenshots, UI Automation, Chrome CDP, keyboard / mouse, terminal — with semantic discover-then-act targeting that avoids pixel-coordinate guessing, and per-action perception guards that catch wrong-window typing before it happens.
npx -y @harusame64/desktop-touch-mcp32 tools, native Rust engine (UIA in 2 ms), zero-config PowerShell fallback, full CJK support, MIT licensed. Add the snippet above to your Claude / Cursor / VS Code Copilot config and Claude can drive Notepad, Excel, Chrome, Windows Terminal, and any other app on your machine.
Why this over pixel-clicking? Two ideas run through every tool: discover-then-act —
desktop_discoverreturns interactive entities with short-lived leases instead of raw coordinates, sodesktop_actoperates on what you mean, not where it was — and per-action perception guards that verify the target window's identity and bounds before input lands, catching wrong-window typing and stale-coordinate clicks before they happen.2.1 reads UI Automation by element count instead of depth: Chrome, Edge and VS Code pages, Explorer and Settings values, and Word's body are read and can be acted on, and what UIA still cannot see falls back to OCR and Set-of-Marks as before.
desktop_actalso types into Word's body, and into Windows Terminal after asking the user. 2.0 refuses an action that cannot be done, with a reason, instead of reporting it done, and an action aimed byhwndreaches that window only. See the CHANGELOG.
Features
🔁 Every act reports what it did —
desktop_actanswers with what changed: elements that appeared or disappeared, a modal, a focus move, the screen's repaint (observation), and withnarrate:"rich"the values and names that changed. The agent reads the result of its click from the reply instead of taking another screenshot. On UIA-blind targets it can also attach a PNG of just the region that changed (roiCapture; on by default for a visible change,returnCapture:"never"to suppress,"always"to force).🛑 An act that cannot be done is refused, not reported as done — input that would not reach the field, a window blocked by a dialog, a closed window, a window on another virtual desktop: nothing is sent, and the reply names the reason and what to do next. An act aimed by
hwndnever lands in another window with the same title.🌐 Reads deep windows (2.1) — The UI Automation read goes to depth 64 and 500 elements. Chrome, Edge and VS Code pages, which 2.0 called blind, Explorer and Settings values it left out, and Word's body are read and can be acted on (a Chrome page: 6 elements in 387 ms → 45 in 186 ms).
🎯 Set-of-Marks (SoM) visual fallback — Games, RDP sessions and apps with no accessibility tree still return clickable elements: when UIA is blind,
desktop_discoverandscreenshot(detail="text")switch to a Hybrid Non-CDP pipeline — Rust-powered grayscale + bilinear upscale → Windows OCR → clustering → red bounding boxes with numbered badges ([1],[2]…). Two representations come back: a PNG for spatial orientation and anelements[]list withclickAtcoords — no CDP required.⌨️ Types where background input does not reach — Windows Terminal, after asking the user each time (needs an MCP client with elicitation, over stdio), and Word's body, at Word's caret whether it is in front or behind.
🔐 Key Locker — the terminal autofills your SSH / sudo passwords — Save a credential once into the locker's own secure dialog (stored encrypted on your machine with Windows DPAPI; never shown to the assistant), then run
ssh/sudoin a console opened bykey_locker(action='launch_console')— the password is filled in automatically when the hidden prompt appears, with a per-fill confirmation prompt by default. See Key Locker.⚡ Rust native core — The UIA bridge and image diffing are a Rust native addon (
napi-rs+windows-rs): UIA is called over COM from a dedicated thread instead of spawning PowerShell, and image diffs use SSE2 SIMD. Without the addon, every function falls back to PowerShell transparently. The npm launcher fetches only the GitHub Release matching its version and verifies the Windows runtime zip before extracting it.LLM-native design — Built around how LLMs think, not how humans click.
run_macrobatches multiple operations into a single API call;diffModesends only the windows that changed since the last frame. Minimal tokens, minimal round-trips.Reactive Perception Graph — Register a
lensIdfor a window or browser tab, pass it to action tools, and get guard-checkedpost.perceptionfeedback after each action. It reduces repeatedscreenshot/desktop_statecalls and prevents wrong-window typing or stale-coordinate clicks.Full CJK support — Uses Win32
GetWindowTextWfor window titles, avoiding nut-js garbling. IME bypass input supported for Japanese/Chinese/Korean environments.3-tier token reduction —
detail="image"(~443 tok) /detail="text"(~100–300 tok) /diffMode=true(~160 tok). Send pixels only when you actually need to see them.1:1 coordinate mode —
dotByDot=truecaptures at native resolution (WebP). Image pixel = screen coordinate — no scale math needed. Withorigin+scalepassed tomouse_click, the server converts coords for you — eliminating off-by-one / scale bugs.Browser capture data reduction —
grayscale=true(~50% size),dotByDotMaxDimension=1280(auto-scaled with coord preservation), andwindowTitle + regionsub-crops help exclude browser chrome and other irrelevant pixels. Typical reduction for heavy captures: 50–70%.Chromium smart fallback —
detail="text"on Chrome/Edge/Brave auto-skips UIA (prohibitively slow there) and runs Windows OCR.hints.chromiumGuard+hints.ocrFallbackFiredflag the path taken.UIA element extraction —
detail="text"returns button names andclickAtcoords as JSON. Claude can click the right element without ever looking at a screenshot.Auto-dock CLI —
window_dock(action='dock')snaps any window to a screen corner with always-on-top. SetDESKTOP_TOUCH_DOCK_TITLE='@parent'to auto-dock the terminal hosting Claude on MCP startup — the process-tree walker finds the right window regardless of title.Emergency stop (Failsafe) — Park the mouse in the top-left corner of the primary monitor (within 10px of 0,0) for 500ms to trigger the emergency stop.
Related MCP server: pywinauto-mcp
Requirements
OS | Windows 10 / 11 (64-bit). macOS 14+ on Apple Silicon: limited preview, 4 tools only — see macOS (preview) |
Node.js | v20+ recommended (tested on v22+) — **to develop or run the test suite, `^22.12 |
PowerShell | 5.1+ (bundled with Windows) — used only as fallback when the Rust native engine is unavailable |
Claude CLI |
|
Note: nut-js native bindings require the Visual C++ Redistributable. Download from Microsoft if not already installed.
Note (Key Locker): The credential helper Key Locker uses is an unsigned executable, so on some machines Windows SmartScreen or antivirus may show an "unknown publisher" warning the first time it runs. This is expected — the helper ships with desktop-touch-mcp and runs locally on your machine; you can allow it to proceed. Code signing is planned for a future release.
Installation
npx -y @harusame64/desktop-touch-mcpThe npm launcher resolves runtime strictly by npm package version. For package X.Y.Z, it fetches only GitHub Release tag vX.Y.Z, downloads the zip for your platform (desktop-touch-mcp-windows.zip, or desktop-touch-mcp-macos-arm64.zip on an Apple Silicon Mac), verifies its SHA256 digest, and only then expands it under %USERPROFILE%\.desktop-touch-mcp (~/.desktop-touch-mcp on macOS). Verified cached releases are reused on later runs.
Set DESKTOP_TOUCH_MCP_HOME to override the cache root directory.
On a shared or CI network? The first run reads the GitHub Releases API to locate the runtime zip. The anonymous limit is 60 requests/hour per IP, which a shared public address (CI runners, office NAT) can exhaust before your download even starts. Set
GITHUB_TOKEN(orGH_TOKEN) in the environment and the launcher authenticates the request, raising the limit to 5,000 requests/hour. No token is needed on an ordinary home connection.
Running the launcher from a source checkout? A source build's
bin/launcher.jscarries a placeholder integrity hash (sha256: "PENDING") instead of a finalized one. Rather than download and run an unverified runtime, the launcher fails closed — this guard stops an accidentally published or unfinalized launcher from silently starting unverified code. Published npm releases always ship a real SHA256, so end users never see this. If you are intentionally running the launcher from source, setDESKTOP_TOUCH_MCP_ALLOW_UNVERIFIED=1to skip integrity verification (development only).
Does your host give up before the launcher finishes? Some desktop hosts allow a plugin a fixed budget — 60 seconds is common — to become ready, and a launcher waiting on an unreachable GitHub can spend all of it. Two environment variables cover that case.
DESKTOP_TOUCH_MCP_FETCH_TIMEOUT_MS(default15000) bounds how long the launcher waits without hearing from GitHub. It applies to the release lookup and to the download; for the download it counts silence rather than total time, so a large runtime still installs over a slow connection. A value that is not a positive number of milliseconds is ignored with a warning.
DESKTOP_TOUCH_MCP_OFFLINE_FALLBACK=1lets the launcher start a release that is already installed when GitHub cannot be reached at all. It is off by default. GitHub is always contacted first, so a reachable network still re-downloads and repairs a damaged install; only a network failure reaches the fallback, which starts the copy of your version on disk — without re-verification — or, when that version was never installed, the newest older release that completed a verified install. Answers that are not network failures (a 404, the API rate limit, a mismatched integrity hash) still stop startup loudly. Leave it off unless a host timeout forces your hand: while it is set, a corrupted install of your current version is reused instead of being repaired.The two work together: with the fallback on, startup still waits out the timeout before falling back, so lower
DESKTOP_TOUCH_MCP_FETCH_TIMEOUT_MSif your host's budget is tight. Note also that a download which is still arriving, however slowly, is never interrupted — the fallback answers when the network has gone silent, not when it is merely slow.
Register with Claude CLI
Add to ~/.claude.json under mcpServers:
{
"mcpServers": {
"desktop-touch": {
"type": "stdio",
"command": "npx",
"args": ["-y", "@harusame64/desktop-touch-mcp"]
}
}
}No system prompt needed. The command reference is automatically injected into Claude via the MCP initialize response's instructions field.
Register with other clients (HTTP mode)
Clients that require an HTTP endpoint (GPT Desktop, VS Code Copilot, Cursor, etc.) can use the built-in Streamable HTTP transport:
npx -y @harusame64/desktop-touch-mcp --http
# or with a custom port:
npx -y @harusame64/desktop-touch-mcp --http --port 8080The server starts at http://127.0.0.1:23847/mcp (localhost only). Register the URL in your MCP client settings. A health check is available at http://127.0.0.1:<port>/health.
In HTTP mode the system tray icon shows the active URL and provides quick-copy and open-in-browser shortcuts.
Development install
git clone https://github.com/Harusame64/desktop-touch-mcp.git
cd desktop-touch-mcp
npm installBuild after install:
npm run buildFor a local checkout, register the built server directly:
{
"mcpServers": {
"desktop-touch": {
"type": "stdio",
"command": "node",
"args": ["D:/path/to/desktop-touch-mcp/dist/index.js"]
}
}
}Note: Replace
D:/path/to/desktop-touch-mcpwith the actual path where you cloned this repository.
macOS (preview)
Limited preview. The macOS build has only 4 of the tools — a small subset of the Windows build — and its behaviour may change between releases.
On an Apple Silicon Mac (macOS 14 or later) the same npx command starts a macOS server with four tools: desktop_state, desktop_discover, desktop_act (press, replace or append text) and screenshot (one window). The other tools are Windows-only and are not listed on macOS.
Grant Accessibility — and Screen Recording for window titles and screenshots — to the app that runs the server (Terminal, iTerm, VS Code, the Claude app, …) in System Settings › Privacy & Security. On recent macOS the Accessibility pane can have another name; these commands open the two panes directly:
open "x-apple.systempreferences:com.apple.preference.security?Privacy_Accessibility" open "x-apple.systempreferences:com.apple.preference.security?Privacy_ScreenCapture"Then restart the MCP server — restart or reconnect it in your client, or restart the app that runs it: a server that was already running did not see a new Accessibility grant. If screenshots still fail after granting Screen Recording, quit and reopen that app. Until then the tools answer
PermissionRequiredand say what to grant.Install with
npxonly: a release zip downloaded with a browser is refused by Gatekeeper (the native module is not notarized).stdio only (
--httpis not available yet). Intel Macs and Linux are not supported.
Tools (32 Optimized Tools)
📖 Full Reference:
docs/system-overview.md— Exhaustive guide on parameters, return schemas, and coordinate math.
🌐 World-Graph V2 (Primary Path)
Tool | Description |
| Observe the desktop. Returns interactive entities with leases (UIA, CDP, Terminal, Visual SoM). |
| Perform actions (click, type, set a value, select) on entities via lease validation. Returns semantic diffs — plus an optional |
👁️ Observation & State
Tool | Description |
| Lightweight check of focus, active window, cursor, and Auto-Perception attention signal. |
| Multi-mode capture: |
| Inspect and prune the on-disk screenshot cache behind the by-ref links: |
| Instant session orientation: all window thumbnails + UI summaries in one call. |
| Diagnostic check for native engine health and feature activation. |
⌨️ Input & Control
Tool | Description |
| Send keyboard input. Supports background input (WM_CHAR) and IME-safe clipboard bypass. |
| Precision coordinate-based interaction with homing and force-focus protection. |
| Multi-strategy: |
| Legacy UIA-based click by name/ID (fallback when entities are unavailable). |
🌐 Browser CDP (Chrome/Edge/Brave)
Tool | Description |
| Idempotent debug-mode launch and reliable navigation. |
| High-level DOM interaction stable across repaints and framework re-renders. |
| Deep inspection via |
| Semantic discovery, grep-like DOM search, and pixel-accurate coordinate lookup. |
🛠️ Utilities & Workflow
Tool | Description |
| Unified command execution: |
| Efficient server-side polling for window, focus, text, or URL state changes. |
| Window management: |
| Launch apps and auto-detect new HWNDs (supports localized titles). |
| Batch up to 50 operations into a single round-trip for maximum efficiency. |
| System-level text exchange and user alerts. |
| Manage credentials the terminal autofills for you (SSH key passphrases, sudo / login passwords). Secrets are entered once into the locker's own secure dialog and stored encrypted on this machine (Windows DPAPI); they are never shown to the assistant. |
📊 Office (Excel)
Tool | Description |
| Author and run Excel VBA macros via COM. |
Standard workflow (v1.0.0)
The v2 World-Graph surface (desktop_discover / desktop_act) is the recommended dispatch path. The four-call shape works for native apps, browsers, and terminals identically.
desktop_state → orient: focused window/element, modal, attention signal
desktop_discover → find actionable entities (returns lease + windows[])
desktop_act(lease, …) → act on entity (returns attention + post.perception)
desktop_state → confirm the world changed as expectedClicking — priority order:
browser_click(selector) → Chrome / Edge (CDP, stable across repaints)
desktop_act(lease, action='click') → native / dialog / visual (entity-based; use after desktop_discover)
click_element(name | automationId) → native UIA fallback if desktop_act returns ok:false
mouse_click(x, y, origin?, scale?) → pixel last resort; origin+scale from dotByDot screenshots onlyRecovery hints for each refusal, and how leases expire: Guide → Recovery hints and leases.
More in the guide
The details live in the guide:
Known limitations
Limitation | Detail | Workaround |
Games / video players may return black or hang in PrintWindow capture | DirectX fullscreen apps may not redraw under | Retry with |
UIA call overhead | ~2 ms (focus) / ~100 ms (tree) via Rust native engine; ~300 ms via PowerShell fallback | Rust engine loads automatically; |
| That read stays shallow on Chromium ( |
|
Chromium title-regex misses when sites rewrite | Guard relies on the | Title is treated as plain Chrome (UIA runs). OCR path is still reachable via |
| If Chrome is already running on the default profile without the flag, | Close Chrome first, then |
Layer buffer TTL | Buffer auto-clears after 90s of inactivity → next | After long waits, call |
| When | Call |
| Non-ASCII punctuation (em-dash | Always use |
| Setting | Use |
Thanks
Huge thanks to everyone who tried a desktop-automation MCP server, filed issues, opened PRs, and shared what broke. Every bug report made the next release better. Thank you for building with me!
License
MIT
Available Tools
30 toolsbrowser_clickA
Click a DOM element in Chrome/Edge. Two ways to target: (1) selector — a CSS selector (combines browser_locate + mouse_click; stable across repaints); or (2) by-axis (semantic) — by:'text'|'regex'|'role'|'ariaLabel' + pattern, so you do not have to build a CSS selector for dynamic-class SPAs. by-axis resolves to a SINGLE actionable element (climbing to a clickable ancestor up to 3 levels, hit-testing for occlusion) and STOPS with code:'BrowserAmbiguousTarget' (candidates[] + next[] hints) when 2+ actionable elements match, or code:'BrowserNoActionableTarget' when matches exist but none is clickable — it never guesses. If the target is behind a modal dialog blocking the page, BOTH targeting modes STOP with code:'BrowserModalBlocking' (context.blockingElement {name, role}) instead of clicking through to the backdrop — dismiss the dialog (its close button or Escape) and retry; a plain navigation drawer does not count as blocking. Optionally add role to filter (by:'text',pattern:'Save',role:'button') and scope to narrow the search. Provide EITHER selector OR by+pattern (not both). Pass tabId+port so the server auto-guards (verifies tab readyState and identity) and returns post.perception.status. lensId is optional for advanced pinned-tab workflows. Caveats: selector mode fails if the element is outside the visible viewport — scroll it into view with browser_eval("document.querySelector('sel').scrollIntoView()") first (by-axis only resolves in-viewport actionable targets). hints.verifyDelivery:{status:'delivered'|'unverifiable', reason, observedSignals:{mutationCount,urlChanged,activeElementChanged}} reports the post-click observation in 2 values: 'delivered' fires only when mutationCount>0 OR urlChanged (activeElementChanged is recorded in observedSignals but intentionally NOT a delivery signal — plain clicks on focusable controls always update focus, treating that as 'delivered' would mask silent-fail regressions); 'unverifiable' reason ∈ {'iframe_context_mismatch','no_dom_mutation','probe_install_failed','probe_read_failed'}. CDP emits 2 values only (focus_only is a UIA-path concept, N/A here). BrowserClickNotDelivered is reserved-only (false-positive risk too high to emit) — degradation reads from 'unverifiable' status.
| Name | Required | Description | Default |
|---|---|---|---|
| by | No | Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with `pattern`. The server resolves to a SINGLE actionable element — climbing to a clickable ancestor up to 3 levels — and STOPS with a candidate list (code:'BrowserAmbiguousTarget') if the match is ambiguous; it never guesses. | |
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| role | No | Optional ARIA/implicit-role filter AND-combined with `by` (e.g. by:'text', pattern:'Save', role:'button'). | |
| fixId | No | Approve a pending suggestedFix (one-shot, 15s TTL). Selector mode only. | |
| scope | No | Optional CSS selector to limit the `by`-axis search scope (disambiguation). | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| lensId | No | Optional perception lens ID. Guards (target.identityStable) are evaluated before clicking, and a perception envelope is attached to post.perception on success. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. "rich": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — "@active" and the dialog rescue both resolve one ("ambiguous_title"); or the target moved between the snapshot and the action ("target_changed"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see ("fix_target_unknown") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too ("tree_truncated"). The action itself is unaffected in every case; only the diff is. Default: "minimal". | minimal |
| pattern | No | Value matched against the chosen `by` axis (required when `by` is set). | |
| selector | No | CSS selector for the target element (e.g. '#submit', '.btn'). Provide EITHER selector OR by+pattern. | |
| caseSensitive | No | Case-sensitive matching for by:'text'/'regex' (default false). | |
| scrollIntoView | No | When true, if the target is outside the viewport, scroll it into view (centered) before clicking, instead of failing with ElementNotInViewport. Default false preserves the explicit scrollIntoView-then-retry workflow. Selector mode only (by-axis resolves only in-viewport actionable targets). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden and it does so richly: it discloses the ambiguity/no-actionable-target/modal-blocking stop codes, the ancestor-climb depth and occlusion hit-testing, the auto-guard behavior for tabId+port, and the precise verifyDelivery semantics including why activeElementChanged is deliberately not a delivery signal. This is far beyond what any annotation could convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose and targeting modes are correctly front-loaded, and the error-code content earns its place, but the description is heavily overloaded — the BrowserClickNotDelivered reserved-note, the CDP '2 values only' aside, and cross-tool hwnd/UIA commentary add length without helping an agent call this tool. Several clauses duplicate schema text verbatim.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description assumes responsibility for return-shape disclosure and does it thoroughly: post.perception.status, hints.verifyDelivery with its status enum and reason values, and the observedSignals fields. For a 13-parameter mutation tool with zero annotations, nothing an agent needs to invoke it correctly or interpret the result is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 13 parameters, and much of the description's parameter content (by-axis behavior, ancestor climb) merely restates the schema text. It adds some value by tying parameters together (role as an AND-filter, scope as narrowing, the one-of selector/by constraint) but does not introduce syntax or formats the schema lacks. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Click a DOM element in Chrome/Edge') and immediately disambiguates against siblings by naming browser_locate and mouse_click and explaining how selector mode combines them. An agent can distinguish this from click_element, mouse_click, and browser_locate without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Spells out the two targeting modes and when each applies ('stable across repaints' vs 'dynamic-class SPAs'), the mutual-exclusion rule ('Provide EITHER selector OR by+pattern (not both)'), the modal-blocking fallback (dismiss then retry), and the out-of-viewport workaround via browser_eval. These are explicit when/when-not conditions with named alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_evalA
Purpose: Inspect or operate on a browser tab via 3 actions: 'js' (evaluate JS), 'dom' (get HTML), 'appState' (extract SSR-injected SPA state). Details: action='js' — Run a JS expression. withPerception:true wraps in {ok, result, post}. action='dom' — Return outerHTML of selector (or document.body), truncated to maxLength. action='appState' — Scan Next/Nuxt/Remix/Apollo/GitHub/Redux SSR injected JSON; pass selectors to override defaults. Prefer: Use action='appState' BEFORE 'dom' or 'js' on SPAs where rendered HTML is sparse — single CDP call. Use 'dom' when 'appState' is empty and you need page structure. Use 'js' as the escape hatch for arbitrary scripting. Caveats: DOM nodes cannot be returned from action='js' directly (circular refs are serialized safely). React/Vue/Svelte controlled inputs cannot be set via element.value — use keyboard(action='type') / browser_fill instead. readyState is strictly checked; guard blocks if page is still loading. Typed errors: code:'BrowserNotConnected' on CDP disconnect (re-attach via browser_open); code:'AutoGuardBlocked' when the auto-guard refuses (e.g. page still loading) — the error message preserves the guard's 1-sentence recommended next step (most often wait_until({condition:'ready_state'}) or browser_eval readyState polling, then retry). Examples: browser_eval({action:'js', expression:'document.title'}) → page title browser_eval({action:'dom', selector:'#main', maxLength:5000}) → outerHTML browser_eval({action:'appState'}) → default SPA state probes
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| action | Yes | Action selector — one of: js, dom, appState. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| lensId | No | Optional perception lens ID. Guards (target.identityStable) are evaluated before eval. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| maxBytes | No | Max bytes per individual payload (default 4000). Larger payloads are truncated. | |
| selector | No | CSS selector for root element. Omit for document.body. | |
| maxLength | No | Max characters of HTML to return (default 10000). | |
| selectors | No | Custom probe selectors. Omit to use the default SPA framework set (__NEXT_DATA__ / __NUXT_DATA__ / __REMIX_CONTEXT__ / __APOLLO_STATE__ / window:__INITIAL_STATE__ etc.). Window globals must be prefixed with 'window:'. | |
| expression | No | JavaScript expression to evaluate. The server automatically wraps snippets in an async IIFE to avoid repeated const/let collisions. For multi-statement snippets, use an explicit final return value. Declarations (const/let/var) are scoped per snippet — use window.* / globalThis.* for persistence. A single eval is bounded by the CDP per-command timeout (~15s): do NOT write in-page polling loops here — use wait_until (element_matches / url_matches / ready_state) to wait for conditions instead. | |
| includeContext | No | When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab. | |
| withPerception | No | When true, return structured JSON {ok, result, post} with post.perception attached. Default false preserves raw-text return. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and discharges it: circular-ref serialization, controlled-input restrictions, strict readyState checking, and two typed errors (BrowserNotConnected, AutoGuardBlocked) with their recovery paths. It also notes the ~15s CDP eval bound and that polling loops belong in wait_until.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Labeled sections (Purpose / Details / Prefer / Caveats / Examples) are well front-loaded and skimmable. The Examples block re-states call shapes already implied by the schema, which is mild redundancy, but overall every section contributes.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 12-parameter, no-annotation, no-output-schema tool this covers actions, error taxonomy, alternatives, and caveats thoroughly. The one gap is that return shapes per action are shown only by example rather than described, though withPerception and includeContext hints partially bridge it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real value: it ties parameters to actions (selector/maxLength belong to 'dom', selectors to 'appState'), explains withPerception's {ok, result, post} wrapping, and notes that per-action required fields are enforced at call time despite the flat schema. That last point is a genuine clarification of a schema quirk.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Inspect or operate on a browser tab') and immediately enumerates the three actions (js/dom/appState) with a one-line gloss of each. This distinguishes it from siblings like browser_navigate, browser_click, and browser_locate without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer' block gives explicit routing rules: appState before dom/js on SPAs, dom when appState is empty, js as the escape hatch, and keyboard/browser_fill instead of value-setting on controlled inputs. Alternatives and exclusions are stated, not implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_fillA
Fill a form input with a value via CDP — works on React/Vue/Svelte controlled inputs that reject browser_eval value assignment. Two ways to target: (1) selector — a CSS selector (use browser_overview / browser_locate to find one); or (2) by-axis (semantic) — by:'text'|'regex'|'role'|'ariaLabel' + pattern (e.g. by:'ariaLabel', pattern:'Email address', or by:'role', pattern:'textbox'), so you do not have to build a CSS selector. by-axis resolves to a SINGLE fillable element and STOPS with code:'BrowserAmbiguousTarget' (candidates[] + next[] hints) when 2+ match, or code:'BrowserNoActionableTarget' when the match is not a fillable input/textarea/contenteditable — it never guesses. Optionally add role to filter and scope to narrow. Provide EITHER selector OR by+pattern (not both). Use this over browser_eval when setting a controlled input's value via JS does not update framework state. Caveats: Requires browser_open (CDP active). actual in the response shows the element's value after fill; verify it matches the intended value (for a field the page masks, such as a password, neither the value you passed nor the one read back is echoed — valueWithheld:'masked' — and the match is still checked, in the page). Typed errors: code:'BrowserFillNotDelivered' on post-fill value mismatch — note the false-positive case where a React controlled input's onChange transforms the value (delivery actually succeeded; hints.verifyDelivery.subReason:'controlled_input_transform' for that case; the actual value is authoritative).
| Name | Required | Description | Default |
|---|---|---|---|
| by | No | Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with `pattern`. The server resolves to a SINGLE actionable element — climbing to a clickable ancestor up to 3 levels — and STOPS with a candidate list (code:'BrowserAmbiguousTarget') if the match is ambiguous; it never guesses. | |
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| role | No | Optional ARIA/implicit-role filter AND-combined with `by` (e.g. by:'text', pattern:'Save', role:'button'). | |
| scope | No | Optional CSS selector to limit the `by`-axis search scope (disambiguation). | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| value | Yes | Text to fill into the input element | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| pattern | No | Value matched against the chosen `by` axis (required when `by` is set). | |
| selector | No | CSS selector for the input element. Provide EITHER selector OR by+pattern. | |
| caseSensitive | No | Case-sensitive matching for by:'text'/'regex' (default false). | |
| includeContext | No | When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does so: it discloses ambiguity handling (stops with BrowserAmbiguousTarget plus candidates[]/next[] hints), non-fillable targets (BrowserNoActionableTarget), never guessing, the BrowserFillNotDelivered error, the controlled_input_transform false positive, masked-field behavior (valueWithheld:'masked'), and the CDP/browser_open requirement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose is front-loaded in the first clause and nearly every sentence carries non-redundant operational detail (errors, caveats, verification). It is a dense wall of nested parentheticals, though, and would scan better as bullets — the length is earned but the structure is heavy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 11-parameter, no-annotation, no-output-schema mutation tool, the description supplies what the structured fields lack: error taxonomy, verification semantics, masking caveat, and prerequisite. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 100%, so the baseline is 3; the description adds cross-parameter meaning by restating the selector-vs-by+pattern mutual exclusion and explaining what the by-axis resolution does to a match. It stops short of adding syntax or format detail beyond the schema's own enum and pattern documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Fill a form input with a value via CDP') and immediately scopes the capability to React/Vue/Svelte controlled inputs that reject JS value assignment. An agent can distinguish it from browser_eval, browser_click, and browser_form without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names the alternative and the selecting condition: 'Use this over browser_eval when setting a controlled input's value via JS does not update framework state.' It also routes to browser_overview/browser_locate for selector discovery, states the EITHER selector OR by+pattern exclusivity, and notes the browser_open prerequisite.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_formA
Inspect all form fields (input, select, textarea, button) within a CSS-selector-specified container and return their name, type, id, current value (withheld for a field the page masks, such as a password: value is null, valueWithheld is 'masked', and hasValue says whether it holds one), hint text, disabled/readOnly state, and associated label text (resolved via for[id], ancestor LABEL, aria-labelledby, aria-label in that order). Use this before browser_fill to discover exact field selectors and avoid accidentally targeting the wrong input (e.g. a global search bar). Caveats: Requires browser_open (CDP active). Hidden inputs (type=hidden) are excluded by default — set includeHidden:true if needed. Value text is truncated at 200 chars.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| selector | Yes | CSS selector for the form or container element to inspect (e.g. '#login-form', '.search-bar'). All input, select, textarea, and button descendants are returned. | |
| maxResults | No | Maximum number of form fields to return (default 100). | |
| includeHidden | No | When true, include hidden inputs (type=hidden). Default false to avoid CSRF-token / serialized-state clutter. | |
| includeContext | No | When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations the description carries the full burden and delivers: masking semantics (value null, valueWithheld 'masked', hasValue), label resolution precedence (for[id] > ancestor LABEL > aria-labelledby > aria-label), hidden inputs excluded by default with an opt-in, and 200-char value truncation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single dense paragraph, front-loaded with the action and return shape, then usage, then caveats — every sentence earns its place. It is long, but the length is justified by the return-field enumeration and required by the absence of an output schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter read tool with no annotations and no output schema, the description fully covers what is returned, the masking/truncation edge cases, the dependency, and the workflow position. Nothing needed to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description nonetheless adds value beyond the schema by explaining the masking fields and truncation behavior tied to the value output, plus the rationale for includeHidden. It does not extend the meaning of port, tabId, include, or maxResults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (inspect) and resource (form fields: input, select, textarea, button) scoped to a CSS-selector container, and enumerates the returned attributes. An agent can distinguish it from browser_fill, browser_locate, or browser_search without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to use it before browser_fill to discover exact field selectors and to avoid targeting the wrong input (naming a concrete failure mode, a global search bar). It also states the hard prerequisite (browser_open / CDP active).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_locateA
Find a DOM element by CSS selector and return its physical screen coordinates — compatible directly with mouse_click. Prefer browser_click to find+click in one step. Prefer browser_overview to discover selectors. Caveats: Coordinates are captured at call time; if the page reflows before mouse_click, coords may be stale. Typed errors: code:'BrowserNotConnected' (call browser_open first), code:'ElementNotFound' (selector did not match — re-discover via browser_overview / browser_search).
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| selector | Yes | CSS selector for the target element (e.g. '#submit', '.btn', 'button[type=submit]') | |
| includeContext | No | When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden and does so well: it discloses the staleness caveat (coords captured at call time, may go stale after reflow), the prerequisite (call browser_open on BrowserNotConnected), and two typed error codes with recovery paths. This is exactly the behavioral context a caller needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and output, then alternatives, then caveats, then error codes. Every sentence carries actionable information; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description still covers return shape (screen coordinates usable by mouse_click), prerequisites, failure modes, and recovery steps. Nothing needed to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the schema already documents all five parameters thoroughly (port default, tabId sourcing, include envelope/raw semantics, includeContext token-saving). The description adds no parameter-level detail beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Find a DOM element by CSS selector') plus the exact output ('physical screen coordinates'). It also names the sibling relationships (compatible with mouse_click), so an agent can distinguish it from browser_click and browser_eval without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes to alternatives: 'Prefer browser_click to find+click in one step' and 'Prefer browser_overview to discover selectors.' Both the general case and the discovery case are covered, with the selecting condition stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_openA
Connect to Chrome/Edge running with --remote-debugging-port and return open tab IDs — required before all other browser_* tools. Pass launch:{} (or with overrides) to auto-spawn a debug-mode browser when no CDP endpoint is live (idempotent: an already-running endpoint is preferred). Returns tabs[] with id, url, title, active — pass tabId to browser_* tools to target a specific tab. Caveats: CDP connection is per-process; if Chrome restarts, call browser_open again to get fresh tab IDs. A Chrome session started without --remote-debugging-port cannot be taken over — close it first or use a separate userDataDir. If the CDP endpoint is unreachable and launch is omitted, returns ok:false (typically code:'BrowserNotConnected' when the fetch surfaces ECONNREFUSED, otherwise code:'ToolError' with error 'Cannot reach Chrome/Edge CDP...'); re-call with launch:{} (idempotent) to auto-spawn or start Chrome manually with --remote-debugging-port=9222.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| launch | No | If set, spawn a debug-mode browser when no CDP endpoint is live on the target port (idempotent: an already-running endpoint is preferred and the spawn step is skipped). Pass {} to use defaults (chrome, C:\tmp\cdp, no initial URL). Omit to perform pure connect. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden and does so: it discloses the per-process CDP lifetime, that restarts invalidate tab IDs, that a non-debug session cannot be taken over, idempotency of launch, and the concrete failure codes (BrowserNotConnected vs ToolError) with recovery steps. This is behavior far beyond what the schema declares.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and prerequisite are front-loaded in the first sentence, and the caveat/error material is grouped at the end. It is dense and somewhat long, and portions of the launch-idempotency wording repeat the schema, but the length is largely justified by the missing annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description supplies the return shape (tabs[] with id/url/title/active), the failure shape (ok:false with code), and the lifecycle caveats. An agent has everything needed to call this correctly and to recover from an unreachable endpoint.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the structured fields already document port, launch sub-fields and include; baseline is 3. The description adds useful framing beyond that by explaining which launch mode is preferred when an endpoint is live and that omitting launch means pure connect, but it does not extend meaning much further than the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Connect to Chrome/Edge running with --remote-debugging-port and return open tab IDs') and immediately distinguishes itself from siblings by declaring it is the prerequisite for all other browser_* tools. An agent can identify its role without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use it ('required before all other browser_* tools'), when to pass launch ({}) vs omit it (pure connect), and what to do in the failure path (re-call with launch:{} or start Chrome manually). The alternative-recovery guidance is unusually complete.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_overviewA
List all interactive elements (links, buttons, inputs, ARIA controls) on the current page with CSS selectors, visible text (an input's name — never what is typed in it), and viewport status — use before browser_click to discover stable selectors, and prefer this over screenshot when verifying button/toggle state after submission (no image tokens, structured output). scope limits to a CSS subsection (e.g. '.sidebar'). Returns state (checked/pressed/selected/expanded) for ARIA custom controls. Also returns a modal: section — whether a true modal dialog is blocking the page (isModal + blocker {name, role} + the signals it was judged on); it is ALWAYS present (isModal:false when no modal), and a navigation drawer is NOT reported as a modal (only an aria-modal / alertdialog / native showModal dialog, or a backdrop-backed dialog that locks the page, is treated as modal). Caveats: Selectors are CDP-generated snapshots — re-call after page navigates or re-renders. An input's text is its name — from aria-labelledby, aria-label, its , title, then its hint text — never what is typed in it: read values with browser_form, which withholds the value of a field the page masks (a password). Typed errors: code:'BrowserNotConnected' (CDP not attached — call browser_open or browser_open({launch:{}})). Note: a non-matching scope CSS selector silently falls back to the full document (does not raise an error) — verify the selector via browser_eval if scoped enumeration is required.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| scope | No | CSS selector to limit the search scope (e.g. '.s-main-slot', '#nav-search-form'). Omit to scan the full page. | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| types | No | Element types to include. Default 'all' returns links, buttons, and inputs. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| maxResults | No | Maximum number of elements to return (default 50). | |
| inViewportOnly | No | When true, only return elements currently visible in the viewport. | |
| includeContext | No | When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does so: it discloses that selectors are CDP snapshots needing re-call after navigation, that input text is the field name and never the typed value, that the modal section is always present and drawers are not modals, that a non-matching scope silently falls back to the full document, and the typed error BrowserNotConnected with its remedy.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded and every sentence carries operational information (alternative routing, caveats, error handling). It is a dense single-paragraph wall of text that would scan better with structure, but there is little filler to remove.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and 8 parameters, the description compensates by describing return content in detail: per-element CSS selectors, text, viewport status, ARIA state, and the always-present modal section with isModal, blocker, and the signals it was judged on. Nothing needed to call it correctly appears missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real semantic value on top: it explains scope limits enumeration to a CSS subsection and warns that a non-matching scope selector silently falls back to the full document rather than erroring. It does not add much on port, tabId, maxResults, or include beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List all interactive elements... on the current page') and enumerates the returned fields (CSS selectors, visible text, viewport status, ARIA state). It directly distinguishes itself from siblings by naming browser_click, screenshot, browser_form, and browser_eval as different tools for different jobs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use is given ('use before browser_click to discover stable selectors') plus a named alternative and the condition that selects it ('prefer this over screenshot when verifying button/toggle state after submission'). It also routes value-reads to browser_form and scope-verification to browser_eval.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_searchA
Grep-like element search across the current page. by: 'text' (literal substring), 'regex', 'role', 'ariaLabel', 'selector' (CSS). Returns results[] sorted by confidence descending — pass results[0].selector to browser_click. Pagination via offset/maxResults. Caveats: Use browser_overview for broad discovery; use browser_search when you know specific text or role to target. Typed errors: code:'BrowserSearchNoResults' (broaden the by:'text' substring or relax the by:'regex' pattern; switch to browser_overview to enumerate selectors), code:'BrowserSearchTimeout' (reduce maxResults / narrow scope), code:'ScopeNotFound' (the scope CSS selector did not match — verify the selector or omit scope), code:'BrowserNotConnected' (call browser_open first, or browser_open({launch:{}}) to auto-spawn).
| Name | Required | Description | Default |
|---|---|---|---|
| by | Yes | Search axis: text/regex/role/ariaLabel/selector | |
| port | No | Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json) | |
| scope | No | CSS selector to limit the search scope. | |
| tabId | No | Tab ID from browser_open. Omit to use the first page tab. | |
| offset | No | Offset into the result set (default 0). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| pattern | Yes | Pattern to match against the chosen axis. | |
| maxResults | No | Max results returned (default 50). | |
| visibleOnly | No | Only visible elements (default true). Set false to include hidden ones with confidence penalty. | |
| caseSensitive | No | Case-sensitive matching for text/regex (default false). | |
| inViewportOnly | No | Only currently-in-viewport elements (default false). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so: it discloses result ordering ('sorted by confidence descending'), the chaining contract ('pass results[0].selector to browser_click'), pagination behavior, and a catalog of typed error codes with remediation. This is behavior an agent could not infer from the schema alone.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose and the distinguishing sibling comparison, then moves to output shape, pagination, and error handling. It is dense and mostly earns its length, though the error-code enumeration is long enough to risk burying the primary usage rule.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description must characterize the return value — and it does ('results[] sorted by confidence descending', results[0].selector). For an 11-parameter tool, it covers purpose, alternatives, chaining, pagination, and failure modes without leaving gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds meaning beyond it — explaining the 'by' axes inline, the offset/maxResults pagination pair, and how results feed into browser_click. It does not add much detail on visibleOnly or caseSensitive beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource: 'Grep-like element search across the current page', immediately conveying the operation and its scope. It explicitly positions itself against sibling browser_overview, so an agent can distinguish the two without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
States the selection rule directly: 'Use browser_overview for broad discovery; use browser_search when you know specific text or role to target.' It also prescribes fallback behavior per error (broaden substring, relax regex, switch to browser_overview to enumerate selectors), giving explicit when-to-use and when-to-switch guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
click_elementA
Invoke a UI element by name or automationId via UIA InvokePattern — no screen coordinates needed. The server auto-guards using windowTitle (verifies identity, foreground, modal) and returns post.perception.status. Prefer over mouse_click for buttons, menu items, and links in native Windows apps. Use desktop_discover first to discover automationIds. Pass fixId from a suggestedFix to re-target after window identity drift. lensId is optional for advanced pinned-lens use. Caveats: Typed errors: code:'InvokePatternNotSupported' — the control does not expose InvokePattern, fall back to mouse_click; code:'ElementDisabled' — the element is in a disabled state, re-check preconditions before retry; code:'GuardFailed' — read the perception envelope (attention / guard fields) and choose recovery (re-focus, wait, or pass the suggestedFix.fixId on the next call). Some custom controls do not expose InvokePattern at all; fall back to mouse_click for those.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). String to avoid 64-bit precision issues. | |
| name | No | Element name/label (partial match, case-insensitive) | |
| fixId | No | Approve a pending suggestedFix (one-shot, 15s TTL). | |
| lensId | No | Optional perception lens ID. Guards (safe.keyboardTarget, target.identityStable) are evaluated before clicking, and a perception envelope is attached to post.perception on success. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. "rich": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — "@active" and the dialog rescue both resolve one ("ambiguous_title"); or the target moved between the snapshot and the action ("target_changed"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see ("fix_target_unknown") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too ("tree_truncated"). The action itself is unaffected in every case; only the diff is. Default: "minimal". | minimal |
| controlType | No | Control type filter, e.g. 'Button', 'MenuItem' | |
| windowTitle | Yes | Partial window title of the target window. Use '@active' for the current foreground window. | |
| automationId | No | Exact AutomationId of the element |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden — and it does disclose meaningful behavior: auto-guarding on windowTitle (identity/foreground/modal), the perception envelope return, and typed error codes with recovery paths. However, it omits the return shape beyond post.perception.status and never states permission or side-effect profile beyond 'invoke', leaving some behavioral burden unmet for a no-annotation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and routing in the first two sentences, with caveats last. It is long, but nearly every clause is actionable (error codes, recovery choice, fallback), so little is filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by describing the perception envelope and post.rich diff behavior, plus typed errors and recovery steps. For a 9-parameter invocation tool with no annotations, this is complete enough to call correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning beyond the schema: windowTitle triggers identity/foreground/modal guarding, fixId re-targets after window-identity drift, and lensId is scoped to advanced pinned-lens use. This is more than the raw schema conveys.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (invoke) and resource (UI element by name or automationId) with the mechanism (UIA InvokePattern) and the constraint (no screen coordinates). It explicitly distinguishes itself from mouse_click, so an agent can route between them without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use (buttons, menu items, links in native Windows apps) and when-not ('Prefer over mouse_click'), plus a workflow prerequisite ('Use desktop_discover first to discover automationIds') and a fallback rule for unsupported controls.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clipboardA
Read or write the Windows clipboard. action='read' returns current text content (empty string if non-text). action='write' replaces clipboard with given text and verifies delivery by reading the clipboard back and comparing the bytes (UTF-16LE) for exact equality. Caveats: Non-text clipboard payloads (images, files) return empty string on read. Reading text larger than about 8MB (roughly 4 million characters, far above anything this tool can write) is refused immediately rather than copied. Calls give up after 4s (read) / 5s (write) — the usual cause is that the application owning the clipboard has stopped responding, which blocks the clipboard for every application on the machine, so retrying does not help until it recovers or is closed. A write that gives up leaves the clipboard in an indeterminate state (it may still complete once that application recovers), unlike code:'ClipboardWriteNotDelivered' where the write is known not to have landed — re-read before relying on the contents. Overwrites existing clipboard content on write. action='write' delivery-verification failure returns code:'ClipboardWriteNotDelivered' — typical causes: a third-party clipboard manager intercepts SetClipboardData, DLP / endpoint protection blocks the payload, RDP / Citrix clipboard transcoding strips the text, or another process clears the clipboard between Set and the read-back. Recovery: retry the write, or fall back to keyboard(action='type', use_clipboard=false) for short text. On builds without the native addon (backend:'powershell') writes are additionally capped at about 12000 characters and return code:'ClipboardWriteTooLargeForFallback' above it. Diagnostics: every response reports backend:'native'|'powershell' — the implementation that served the call. A successful write adds postCloseChecked: whether the read that catches a clipboard manager swapping the payload actually ran (native uses a separate second read; powershell's single read-back is that read), plus postCloseSkipReason when it did not run. On backend:'native' only, a successful write also reports sequenceAfterWrite (a Windows clipboard sequence number, for diagnosis only — the delivery verdict is always the byte comparison) and, when the post-close read alone confirmed the write, inSessionReadable:false. Examples: clipboard({action:'write', text:'hello'}) → write+verify; clipboard({action:'read'}) → returns current text.
| Name | Required | Description | Default |
|---|---|---|---|
| text | No | Text to place on the clipboard | |
| action | Yes | Action selector — one of: read, write. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the entire behavioral burden and does so thoroughly: byte-level (UTF-16LE) write verification, 4s/5s timeouts and why retrying is futile, indeterminate vs known-failed write states, distinct error codes, non-text payload behavior, and the backend-specific 12000-char fallback cap. This is exactly the context an agent needs to interpret results and avoid blind retries.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The first two sentences are well front-loaded, but the body becomes a very long run-on covering diagnostics like sequenceAfterWrite, postCloseChecked, and inSessionReadable that are closer to release notes than call guidance. For a 2-action, 3-parameter tool this is over-sized even though most individual clauses are substantive.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description fully specifies return shapes, per-response fields (backend, postCloseChecked, sequenceAfterWrite), and error codes, plus caveats and recovery. Nothing an agent needs to call this correctly or interpret its result is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds semantics the schema does not: what action='read' actually returns (empty string for non-text/oversize), that write replaces existing content, and that effective text limits differ by backend (12000 chars on powershell vs the schema's nominal 100000 maxLength). That extra meaning lifts it above baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource: 'Read or write the Windows clipboard,' then immediately disambiguates the two modes by their observable effect (read returns text, write replaces and verifies). An agent can distinguish this from the keyboard sibling purely from the description.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit recovery routing — retry the write or fall back to keyboard(action='type', use_clipboard=false) for short text — and specifies when reads of oversize text are refused. It is strong on failure-path guidance, though it never states the general condition for preferring clipboard over other input methods in the first place.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
desktop_stateA
Purpose: Read-only observation of the current desktop state. Returns focused window/element, modal flag, attention signal from Auto Perception. Phase 4 absorbs former get_active_window / get_cursor_position / get_screen_info / get_document_state via include* flags. Details: Always returns: focusedWindow (title, hwnd, processName), focusedElement (name, type, value, automationId), cursorPos {x,y}, cursorOverElement (name, type), cursorOverWindow, hasModal (boolean), pageState ('ready'|'loading'|'dialog'), attention, visibleWindows (windows on screen: not minimised, not on another virtual desktop). Optional fields (default off): includeCursor:true → cursor {x,y,monitorId} (richer than cursorPos). includeScreen:true → screen {virtualScreen, displays[], displayCount, primaryIndex}. includeDocument:true → document {url, title, readyState, selection, scroll, viewport} via CDP (silently omitted on non-Chromium foreground). includeSessionContext:true (or include:['sessionContext']) → sessionContext {origin, consoleSessionId, sessionLabel, sessionState, ownWinStation} for Terminal Services session classification (ADR-017, observability-only). Chromium: cursorOverElement is null (UIA sparse); focusedElement may fall back to CDP document.activeElement; hints.focusedElementSource reports which path produced the row ('view' = engine-perception latest_focus, 'uia' = direct UIA query, 'cdp' = document.activeElement). Does NOT enumerate descendants — use desktop_discover for actionable entity list and window list. Prefer: Use after each action to confirm state. Cheapest observation tool — cheaper than any screenshot. attention='ok' means safe to proceed; other values require recovery (see suggest[]). Set include* flags only when you need the extra data (each adds one syscall or CDP round-trip). Caveats: Cannot detect non-UIA elements (custom-drawn UIs, game overlays). hasModal is a heuristic: a window whose title reads like a dialog (Save As, Error, 確認 …), on screen or holding an on-screen owner disabled — browser alert/confirm dialogs do not appear here. includeDocument requires browser_open (CDP active); silently omitted otherwise with hints.documentUnavailable. focusedElement.value is the focused field's current text, so a plain credential field's value comes back like any other. Masked fields are withheld on the CDP road by rule; on the UIA road the value is whatever the provider serves and nothing here checks for a masked control — a masked field can arrive as mask characters, one per character of the secret, which nothing here distinguishes from its real value. It is also not always present, and hints.focusedElementValueAbsent says why when it is not: 'view_road_has_no_value' (the perception view is preferred and carries no values at all) or 'masked_on_this_road' (the CDP read dropped a masked field). No hint means nothing was dropped on the road that answered — on the UIA road an absent value can still be a field whose provider will not serve one.
| Name | Required | Description | Default |
|---|---|---|---|
| port | No | CDP port for includeDocument. Default: the port the tab named by tabId was opened on, else 9222. | |
| tabId | No | Optional CDP tab id for includeDocument; omit for the focused tab. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| includeCursor | No | When true, add a richer `cursor` field with monitor index alongside the lightweight `cursorPos`. Phase 4: absorbs former get_cursor_position. Default false. | |
| includeScreen | No | When true, add a `screen` field with all connected display info (resolution, position, DPI, scale). Phase 4: absorbs former get_screen_info. Default false. Use the displayId values returned here in screenshot / window_dock(action='dock'). | |
| includeDocument | No | When true, add a `document` field with the focused Chrome tab's url, title, readyState, selection, and scroll position via CDP. Phase 4: absorbs former get_document_state. Default false. Requires browser_open (CDP active); silently omitted on non-Chromium foreground. | |
| includeSessionContext | No | When true, add a `sessionContext` field with the Terminal Services session classification (origin, consoleSessionId, sessionLabel: 'console'|'rdp'|'other', sessionState: 'active'|'connected'|'disconnected'|'locked'|'unknown', ownWinStation). Default false. Equivalent to `include: ['sessionContext']`. Per ADR-017: observability-only — does not gate input. `sessionState: 'locked'` is a heuristic (active + foreground=null + previous sample within 60s saw a non-null foreground); treat it as a generic input-pause signal — it can also fire on secure-desktop transitions (UAC prompt, Credential UI), where the user-visible state is not strictly 'locked' but input is equally unavailable to this session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so: it declares the read-only safety profile, discloses relative cost, and details caveats (cannot detect non-UIA elements, hasModal is a title-based heuristic, includeDocument requires browser_open). The masked-field/value-absence discussion openly flags a security-relevant limitation rather than hiding it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with labeled sections (Purpose / Details / Prefer / Caveats) and leads with the core statement, so it is skimmable. It is nonetheless very long, and the extended masked-field passage is repetitive enough to dilute the section's value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex hub tool with no output schema and no annotations, the description fully compensates: it enumerates always-present and opt-in return fields, explains the Chromium/UIA perception paths and hints, and lists caveats. An agent has everything needed to call it and interpret results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, but the description adds meaning beyond the schema: it explains the cost of each include* flag ('each adds one syscall or CDP round-trip') and the fallback behavior for includeDocument. Most flag semantics are, however, repeated from the schema rather than extended.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Read-only observation of the current desktop state') and enumerates the exact return surface (focusedWindow, focusedElement, cursorPos, hasModal, pageState, attention, visibleWindows). It also explicitly distinguishes itself from siblings: 'Does NOT enumerate descendants — use desktop_discover for actionable entity list and window list.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Prefer: Use after each action to confirm state. Cheapest observation tool — cheaper than any screenshot' gives explicit when-to-use guidance. It also names the alternative (desktop_discover) for enumeration and states when to set the include* flags ('only when you need the extra data'), covering when-not to use the optional paths.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
excelA
Purpose: Author and run VBA macros against Excel via COM late binding (ADR-015). Headline differentiator against Claude for Excel which writes formulas but cannot run VBA.
Details: action='run_vba' authors a Sub in a fresh workbook, saves into the managed Trusted Location (%LOCALAPPDATA%\desktop-touch-mcp\trusted-vba), and Application.Run the macro. Requires HKCU AccessVBOM=1 + VBAWarnings=1 + a registered Trusted Location (all configured by node scripts/enable-access-vbom.mjs). Trust setup: Excel must restart after the CLI runs (values cached at process start). action='check_access_vbom' is a read-only preflight returning {trusted, lockedByPolicy, scope}.
Prefer: Run check_access_vbom first when a workflow depends on macro execution; the remediation hint pre-empts an opaque HRESULT 0x800a03ec failure inside run_vba.
Caveats: macroName MUST appear as Sub <name>(...) in code (else VbaMacroNotFound). VBA Editor UI is structurally bypassed — no UIA tree walk needed. Excel COM is STA: each call serialises through the bridge's worker thread, so long-running macros block subsequent excel() calls on the same MCP server.
Examples:
excel({action:'check_access_vbom'}) → {trusted:true, scope:'hkcu'}
excel({action:'run_vba', code:'Sub DesktopTouchAdHoc()\n Range("A1").Value = "Hello"\nEnd Sub'}) → {ok:true, workbookPath:'...\trusted-vba\dt_vba_.xlsm'}
excel({action:'run_vba', code:'Sub Demo()\n MsgBox "hi"\nEnd Sub', macroName:'Demo', visible:true}) → demo recording path
| Name | Required | Description | Default |
|---|---|---|---|
| code | No | VBA source. MUST declare at least one Sub matching `macroName` (default `DesktopTouchAdHoc`). Example: "Sub DesktopTouchAdHoc()\r\n Range(\"A1\").Value = \"Hello\"\r\nEnd Sub" | |
| action | Yes | Action selector — one of: run_vba, check_access_vbom. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| visible | No | If true, show the Excel window during execution. Default false (headless). Setting visible:true is useful for demo recording but may surface MsgBox/InputBox calls in the macro that block the COM thread. | |
| macroName | No | Sub name to invoke. MUST appear in `code` as `Sub <name>(...)`. Default `DesktopTouchAdHoc`. | DesktopTouchAdHoc |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and delivers: prerequisites (HKCU AccessVBOM=1, VBAWarnings=1, registered Trusted Location, restart-after-CLI), the STA serialization caveat that long macros block subsequent excel() calls, the VbaMacroNotFound failure mode, and the UI-bypass behavior. This is unusually rich behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with 'Purpose:' followed by Details, Prefer, Caveats, and Examples — a predictable structure. It is dense and long, but nearly every line carries non-redundant information, so the length is justified rather than padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex COM/VBA tool with no output schema, the description supplies enough: return-shape examples (trusted/scope, ok/workbookPath), preconditions, and failure modes. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds real meaning: macroName must appear as `Sub <name>(...)` in code (else VbaMacroNotFound), and the three worked examples show how action/code/macroName/visible combine. The cross-parameter constraint between code and macroName is the key added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Author and run VBA macros against Excel via COM late binding') and clearly enumerates the two actions, run_vba and check_access_vbom. It differentiates against an external product ('Claude for Excel') but never distinguishes from the sibling tool run_macro, which an agent must otherwise infer is different.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit ordering rule — run check_access_vbom first when a workflow depends on macro execution — and explains the remediation motivation. It stops short of stating when NOT to use this tool versus siblings like run_macro, so exclusion guidance is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
focus_windowA
Bring a window to the foreground by partial title match (case-insensitive). Use when a tool does not accept a windowTitle param, or when you need to switch focus before a sequence of actions. Use chromeTabUrlContains to activate a specific Chrome/Edge tab by URL substring before focusing — only the active tab's title appears in the windows list. If CDP is unavailable, chromeTabUrlContains is silently skipped — check response.hints.warnings. Returns WindowNotFound if no match exists; call desktop_discover to see available titles. Caveats: On some apps focus may be immediately stolen back (modal dialogs, UAC prompts) — verify with desktop_state after focusing. Win11 foreground refusal (UIPI cross-elevation / admin-only target / call from a background process or service) returns code:'ForegroundRestricted' ok:false instead of silently failing — recover by switching to a tool that does not require foreground transfer: desktop_act / click_element use UIA InvokePattern (no foreground needed); keyboard BG path bypasses foreground for terminal-class targets only (Windows Terminal / cmd / PowerShell — keyboard with windowTitle on non-terminal apps still hits the same ForegroundRestricted refusal). browser_* tools target by tabId/selector, not windowTitle.
| Name | Required | Description | Default |
|---|---|---|---|
| title | Yes | Partial window title to search for (case-insensitive). A shown window (a minimised one counts) is taken before a hidden one with the same text; a hidden one — on another virtual desktop — only when nothing shown matches. | |
| cdpPort | No | CDP port for chromeTabUrlContains (default 9222) | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| forceFocus | No | When set, use AttachThreadInput-based foreground escalation on the first attempt. When omitted (default), focus_window first tries the standard SetForegroundWindow path and auto-escalates to force-focus only if Win11 refused the default attempt (issue #197). Override env: DESKTOP_TOUCH_FORCE_FOCUS=1 sets the implicit default to true. If both default and force paths fail, focus_window now returns ok:false code:'ForegroundRestricted' instead of the previous silent ok:true with windowChanged:false. | |
| chromeTabUrlContains | No | When set, activate the Chrome/Edge tab whose URL contains this substring before focusing the window. Requires Chrome/Edge running with --remote-debugging-port (default 9222). Use this when the target is a Chrome tab that is not currently active — the active tab title is the only one visible in the window title list. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden and does so thoroughly: it discloses silent CDP skipping and where to check (response.hints.warnings), the WindowNotFound error and desktop_discover recovery, focus-stealing caveats, and the Win11 foreground refusal contract (code:'ForegroundRestricted' ok:false instead of silent success).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core verb+resource before any caveats, and nearly every sentence adds operational value. It is dense and long, bordering on caveat overload for a single tool, which keeps it out of the top band.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description still covers error codes, silent-skip semantics, recovery paths, and caveats an agent needs to call this correctly. Nothing material is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real meaning: chromeTabUrlContains activation semantics and why only the active tab's title is visible, plus forceFocus's auto-escalation behavior on Win11. It slightly exceeds what the schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource and matching semantics (foreground a window by case-insensitive partial title). Explicitly contrasts with sibling tools: browser_* target by tabId/selector, not windowTitle, and it names desktop_act/click_element as foreground-free alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use conditions ('when a tool does not accept a windowTitle param', 'before a sequence of actions') plus a pre-condition for the Chrome tab path. Also names concrete alternatives and their selection criteria (UIA InvokePattern, keyboard BG path for terminals only).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
keyboardA
Purpose: Send keyboard input to a window: 'type' for text, 'press' for key combos, 'sequence' for atomic multi-step chords. Details: action='type' inserts text (auto-clipboard for non-ASCII, bypassing IME conversion). action='press' sends key combos like 'ctrl+c'/'alt+tab'. action='sequence' runs ordered steps in one keyboard lock — use for Alt+letter, letter mnemonic chains where intermediate tool calls would close the menu. windowTitle or hwnd is REQUIRED (blank/whitespace counts as neither) — the server focuses and auto-guards that window (identity, foreground, modal) first, and a call with neither stops with DestinationRequired before any key is sent. Use windowTitle:'@active' to aim at the foreground window on purpose; an hwnd naming a titleless window works only while that window is already foreground. DESKTOP_TOUCH_REQUIRE_DESTINATION=0 downgrades the stop to a warning. Prefer: Set lensId for perception guards. Use desktop_act({action:'setValue'}) for UIA ValuePattern text fields. Caveats: win+r/win+x/win+s/win+l blocked. action='type' does not handle CJK IME composition — use use_clipboard=true or desktop_act({action:'setValue'}); neither lands while an IME composition is pending — commit or cancel it first. hints.clipboard reports the backend and whether the clipboard was restored. Non-ASCII text (CJK / emoji / diacritics / smart-quote-class punctuation) auto-clipboards to prevent silent-drop and Chrome accelerator hijack; pass forceKeystrokes:true to disable. Background (PostMessage/WM_CHAR) auto-engages for terminal-class windows (Windows Terminal / cmd / PowerShell); DTM_BG_AUTO=1 enables globally. Foreground non-terminal type runs a per-chunk leash; user focus-steal mid-stream aborts with FocusLostDuringType + context.typed/remaining; pass abortOnFocusLoss:false to disable. BG type verifies WM_CHAR via UIA TextPattern read-back; mismatch returns BackgroundInputNotDelivered (see SUGGESTS for false-positive notes). BG press read-back is scoped to terminal-class + enter/tab/arrow; other combos return verifyDelivery:'unverifiable', failure returns BackgroundKeyNotDelivered. action='sequence' is FG-only (BG/foreground_flash schema-rejected); emits verifyDelivery:'focus_only'; mid-loop focus theft returns MenuFocusLostMidSequence + context.remaining: Step[]. Win11 FG refusal returns ForegroundRestricted — terminal-class targets auto-engage BG (except a combo with ctrl/shift/alt and type with replaceAll, which go through FG and can return ForegroundRestricted too); non-terminal switch to desktop_act / click_element. Off-desktop target → WindowOnOtherDesktop, not sent. Examples: keyboard({action:'type', text:'hello', windowTitle:'Untitled - Notepad'}) → text injected (guarded) keyboard({action:'type', text:'hello', windowTitle:'@active'}) → typed into the foreground window keyboard({action:'press', keys:'ctrl+c', windowTitle:'Untitled - Notepad'}) → copy keyboard({action:'press', keys:'escape', windowTitle:'Dialog'}) → dismiss dialog keyboard({action:'sequence', steps:[{keys:'alt+i', gapMs:100},{keys:'m'}], windowTitle:'Microsoft Visual Basic'}) → Insert > Module (atomic)
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Either this or windowTitle is required. Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues. A window that has no title can be addressed this way, but only while it is already the foreground window — keyboard focus and guarding cannot target a titleless window yet. | |
| keys | No | Key combo string, e.g. 'ctrl+c', 'alt+tab', 'enter', 'ctrl+shift+s'. Note: win+r, win+x, win+s, win+l are blocked for security. | |
| text | No | The text to type (max 10,000 characters) | |
| fixId | No | 'type': Approve a pending suggestedFix (one-shot, 15s TTL). Pass the fixId returned by a previous failed keyboard(action='type') to re-attempt with guard-validated args. 'sequence': Approve a pending suggestedFix (one-shot, 15s TTL). Only meaningful for GUARD-pre-loop rejections (e.g. unsafe.keyboardTarget). Mid-loop MenuFocusLostMidSequence does NOT issue fixIds — recover by re-calling with context.remaining. | |
| steps | No | Ordered list of key-press steps. Min 1, max 16. Total duration must not exceed 5000ms (excludes settleMs and focus acquisition). N=1 is allowed but inherits the sequence verification contract (hints.verifyDelivery.status='focus_only'); if you want the stricter keyboard:press contract, call keyboard({action:'press', keys}) directly (issue #278, matrix doc §3.1). | |
| action | Yes | Action selector — one of: type, press, sequence. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| lensId | No | 'type': Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated before typing, and a perception envelope is attached to post.perception on success. 'press': Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated before the key press. 'sequence': Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated once before the first step. | |
| method | No | 'type' / 'press': Input routing channel. 'auto' uses background (PostMessage) when the target window is a known terminal class (Windows Terminal / cmd / PowerShell) OR DTM_BG_AUTO=1 is set; else foreground. Terminal auto-detect is HWND-targeted so user-side focus changes mid-stream cannot divert keystrokes (except a combo with ctrl/shift/alt, or type with replaceAll, which 'auto' sends through the foreground). 'background' forces PostMessage-only (no focus change, fails on Chromium/IME). A key combo with ctrl, shift or alt, and type's replaceAll (which selects with Ctrl+A), cannot be posted — the app would see the plain key and type it — so 'background' refuses them with BackgroundModifierComboUnsupported and nothing is sent; 'auto' sends them through the foreground. 'foreground' forces the current behavior (SetForegroundWindow + keystrokes). 'foreground_flash' (ADR-013 Option E) is an explicit opt-in 妥協 BG path for Windows Terminal: temporarily steals foreground (~50-80ms), pastes via clipboard, sends Ctrl+V, restores foreground + clipboard. Single-line + < 5KiB only. Carries `typingLeakRisk: true` in hints because user keystrokes during the flash window can leak to WT. Default 'auto'. 'sequence': Sequence is foreground-only by design — Alt-menu mnemonics need real SendInput. Omit, or pass 'foreground'. method:'background' / 'foreground_flash' are rejected at schema parse time (typed codes BackgroundNotApplicableToSequence / ForegroundFlashNotApplicableToSequence document the rationale for LLMs). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. "rich": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — "@active" and the dialog rescue both resolve one ("ambiguous_title"); or the target moved between the snapshot and the action ("target_changed"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see ("fix_target_unknown") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too ("tree_truncated"). The action itself is unaffected in every case; only the diff is. Default: "minimal". | minimal |
| settleMs | No | Milliseconds to wait after the action before checking foreground window (default 300). | |
| forceFocus | No | When true, bypass Windows foreground-stealing protection via AttachThreadInput before focusing the target window. Default: follows env DESKTOP_TOUCH_FORCE_FOCUS (default false). | |
| replaceAll | No | When true, send Ctrl+A to select all existing text before typing. Equivalent to Ctrl+A → keyboard(action='type') in one call (requires field already focused). Default false. | |
| trackFocus | No | When true (default), detect if focus was stolen from the target window after the action. Reports focusLost in the response. Set false to skip. | |
| forceImeOff | No | 'type': Issue #245 系統②: when true, query the target window's IME open-status via Imm32 before typing; if ON, switch OFF for the duration of this call and restore the prior state in `finally`. Prevents silent romaji conversion when the user's Japanese IME is active but the LLM is typing ASCII commands. Requires `windowTitle` or `hwnd` (otherwise no target to query). Default false — existing use_clipboard auto-promotion still handles non-ASCII symbols transparently. No-op when the addon predates the IMM bridge (call proceeds with whatever IME state is in effect). 'sequence': Issue #245 系統②: query the target's IME open-status before the first step; if ON, switch OFF for the whole sequence and restore in finally. Prevents Alt-mnemonic hijack when 日本語 IME is active (the OS routes Alt+letter to IME composition instead of the menu). Requires windowTitle or hwnd. Default false. | |
| windowTitle | No | Partial title of the window that should receive the keystrokes. Required unless you pass hwnd: a call with neither stops with DestinationRequired before any key is sent, and an empty or whitespace-only value counts as neither. The server focuses this window before typing and uses it as the expected target for focusLost detection. Use '@active' to target the current foreground window on purpose. | |
| use_clipboard | No | If true, copy text to clipboard and paste with Ctrl+V instead of simulating keystrokes. Use this when typing URLs, paths, or ASCII text into apps with Japanese IME active — pasted text is not run through IME conversion. Note this does not help while an IME composition is already in progress: the paste keystroke is consumed by the IME and nothing is inserted, though the call still answers ok, so commit or cancel the composition first; when the IME was on at the paste, hints.clipboard.imeOpen says so. Your clipboard is replaced for the duration of the call and put back afterwards; hints.clipboard reports which backend served the paste and whether the restore ran. On builds without the native addon this path is capped at about 12000 characters and fails with code:'ClipboardWriteTooLargeForFallback' above it. Default false. | |
| forceKeystrokes | No | When true, always use keystroke mode even if text contains non-ASCII content (CJK, emoji, diacritics, em-dash, smart quotes, etc.) that would normally trigger auto-clipboard. Default false — auto-clipboard is enabled. | |
| abortOnFocusLoss | No | Focus Leash Phase B: when true, the foreground keystroke send is split into chunks (default 8 chars; override via DTM_LEASH_CHUNK_SIZE env) and the target window's foreground state is verified between chunks. If the user grabs focus mid-stream, the call aborts and returns FocusLostDuringType with context.typed (chars delivered to target) and context.remaining (unsent tail) so the caller can re-focus and retry the unsent portion. Default: true when windowTitle is provided, false otherwise. Has no effect on the clipboard path (atomic Ctrl+V) or the BG (WM_CHAR) path (HWND-targeted, foreground-independent). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden and does so extensively: blocked key combos, IME composition behavior, background versus foreground routing, focus-leash abort behavior, atomic sequence locking, destructive side effects like clipboard replacement, and specific error codes. This is far beyond what structured fields convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with purpose, action modes, required destination, and alternatives, then follows with caveats and examples. It is dense and very long, but the complexity of the tool and the absence of annotations justify most of the detail; some schema-level repetition could be trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 19 parameters, no annotations, no output schema, and a complex windowing/input domain, the description is remarkably complete. It explains destination requirements, failure modes, recovery context, IME handling, focus behavior, and action-specific verification limits, leaving little an agent would need to infer.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaningful semantics beyond the schema by explaining action-specific required fields, when windowTitle versus hwnd matters, why '@active' is special, and how method routing interacts with each action. It does not need to re-document every parameter, but it usefully frames their behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the specific verb ('Send keyboard input') and resource ('to a window'), then immediately distinguishes the three action modes type, press, and sequence. It also routes the agent away from sibling tools by referencing desktop_act({action:'setValue'}) and click_element where appropriate, so the tool's identity is unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit when-to-use guidance for each action mode: type for text, press for key combos, sequence for atomic multi-step menu navigation. It also names alternatives for specialized cases, such as desktop_act for UIA ValuePattern fields, use_clipboard for IME-safe text, and click_element for non-terminal foreground refusal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
key_lockerA
Purpose: Manage credentials the terminal autofills for you (SSH key passphrases, sudo / login passwords). Secrets are entered once into the locker's own secure dialog and stored encrypted on this machine (Windows DPAPI, current user); these stored secrets are NEVER shown to the assistant or sent to this tool — this covers what the locker holds, not every password on the machine.
Details: action='save' pre-seeds a credential for a binding URI (ssh://user@host:22, sudo://host/root, https-cred://host:443, sshkey:SHA256:…): it opens the locker's secure entry dialog and stores the secret; the first save also shows a one-time enable confirmation. action='list' shows saved bindings (metadata only, never secrets). action='forget' deletes a binding and its secret. action='set_policy' toggles per-binding autofill confirmation. action='status' reports whether the locker is enabled (consent) and how many bindings exist. action='launch_console' opens (or reuses) an autofill-capable anchored pane and returns {paneId, windowTitle} — by default a new tab in the user's current Windows Terminal window (host:'classic' opens a dedicated classic console window instead). action='fill_once' asks the user to type a secret into the locker's dialog and types it once into the password prompt (or ssh's 'Enter passphrase for key' prompt) currently shown in a launch_console pane, then forgets it.
Prefer: Autofill is AUTOMATIC when a bound command triggers a credential prompt in the terminal — you never fill a SAVED secret by hand. For a prompt autofill does not handle (an ssh inside another ssh session, or a login the user does not want saved), send the command, then call fill_once on that pane. Autofill and fill_once ONLY work in a pane opened by launch_console (a pre-existing terminal is never autofilled): to autofill, first launch_console, then run the ssh / sudo command with terminal({action:'run'|'send', paneId}) — pass the paneId field, not the windowTitle. Keep the returned paneId; there is no pane-listing action, but launch_console with fresh:false reuses the most-recent pane and returns its paneId again. Use save to enroll, list/status to inspect.
Caveats: Windows-only. The anchored pane defaults to a Windows Terminal tab (autofill and terminal reads operate while that tab is the ACTIVE tab — switching away pauses them safely); host:'classic' opens a dedicated classic console window instead, and is the retry when Windows Terminal is not installed (KeyLockerWtUnavailable). The human can also see and type into the pane. Enabling the locker (first save or launch_console) grants BOTH credential autofill AND the ability for the assistant to launch a locker-owned pane. Disable the whole feature with DESKTOP_TOUCH_DISABLE_KEY_LOCKER=1. An ssh save needs the host key already in known_hosts (connect once first). API-token / env-var credentials are not supported yet.
Examples:
key_locker({action:'status'}) → {consentAccepted:false, disabled:false, bindingCount:0}
key_locker({action:'save', uri:'sudo://buildbox/root'}) → opens the secure dialog → {captured:true}
key_locker({action:'list'}) → {bindings:[{displayUri:'sudo://buildbox/root', scheme:'sudo', …}]}
key_locker({action:'launch_console'}) → {paneId:'wt:31264:13322426700123', windowTitle:'dtm-locker-console-…'} → then terminal({action:'send', paneId:'wt:31264:13322426700123', input:'ssh user@host'})
key_locker({action:'launch_console', host:'classic'}) → {paneId:'12345678', windowTitle:'dtm-locker-console-…'} (dedicated classic console window)
| Name | Required | Description | Default |
|---|---|---|---|
| uri | No | A binding URI identifying the credential target — e.g. `ssh://user@host:22`, `sudo://host/root`, `https-cred://github.com:443`, `sshkey:SHA256:…` (no `//`). Parsed per the L1 grammar; malformed input is rejected. | |
| host | No | 'windows-terminal' (default) = open a new tab in the user's current Windows Terminal window. 'classic' = open a dedicated classic console window instead (the fallback when Windows Terminal is not installed — a KeyLockerWtUnavailable error tells you to retry with this). A Windows Terminal pane autofills/reads only while its tab is the ACTIVE tab; a classic console has its own window. | |
| fresh | No | false (default) = reuse the most-recent still-open anchored pane of the requested host; true = open a NEW one (bounded). | |
| action | Yes | Action selector — one of: list, save, forget, set_policy, status, launch_console, fill_once. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| paneId | No | The paneId returned by launch_console (only those panes can be filled). | |
| reason | No | One short sentence the dialog shows the user as YOUR reason (e.g. 'log in to 192.168.1.20 via the jump host'). Shown as the assistant's words; never put a secret here. | |
| submit | No | true (default) = press Enter after typing the secret; false = type only. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| confirmEveryInjection | No | true (default) = confirm every autofill for this binding; false = opt out (autofill without asking). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so: secrets are never shown to the assistant or sent to the tool, stored via Windows DPAPI for the current user, first save triggers a one-time consent confirmation, enabling grants autofill AND pane-launch capability, disabling via DESKTOP_TOUCH_DISABLE_KEY_LOCKER=1, Windows-only, and ssh saves need the host key already in known_hosts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Long, but organized into Purpose/Details/Prefer/Caveats/Examples with the scoping and safety constraint front-loaded and worked examples at the end. Dense for a single description, though nearly every clause maps to a real invocation decision, so little is expendable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description supplies per-action return shapes and worked call/response examples (including the paneId flow into terminal), covering consent state, bindingCount, and captured/paneId returns. For a 9-parameter, seven-action tool this is complete enough to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (baseline 3), but the description adds real meaning: it explains per-action required fields ('enforced at call time (see the tool description); this flat schema lists every action's fields as optional'), ties paneId to launch_console output, and warns never to put a secret in `reason`.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Manage credentials the terminal autofills for you') and enumerates each action's effect (save pre-seeds a binding, list shows metadata, forget deletes, fill_once types once). It clearly distinguishes itself from the terminal and browser_fill siblings by scoping to SSH/sudo/login credential prompts in a locker-owned pane.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer' block gives explicit routing: autofill is automatic for bound commands, fill_once is for prompts autofill can't handle (nested ssh, unsaved logins), and it names the sibling call to compose with ('then run the ssh / sudo command with terminal({action:'run'|'send', paneId})'). It also states the hard precondition that only launch_console panes can be filled.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
mouse_clickA
Click at screen coordinates. Normally pass windowTitle so the server auto-guards the click (verifies target identity, foreground, coordinate is inside the target rect) and returns post.perception without a confirmation screenshot. origin+scale from dotByDot=true screenshots are converted to screen coords before guarding. doubleClick:true for double-click; tripleClick:true for triple-click (selects a full line of text). Prefer click_element (UIA) for native apps, prefer browser_click for Chrome. Examples: mouse_click({windowTitle:'Notepad', x:200, y:150}) // guarded — post.perception.status='ok'. mouse_click({x:100, y:100}) // unguarded — post.perception.status='unguarded'. If a guard failure returns a suggestedFix, pass its fixId to approve the fix: mouse_click({fixId:'fix-...'}) // one-shot, expires in 15s. lensId is optional and only for advanced pinned-target workflows; omit it for normal use. Caveats: origin+scale are meaningful ONLY with dotByDot=true screenshot responses. hints.verifyDelivery:{status:'delivered'|'focus_only'|'unverifiable', reason} reports the post-click observation; the status is decided from three signals — focused-element shift, the element under the cursor changing between two readable reads, and window-foreground change — or from none of them firing. Win11 foreground refusal during the homing path (UIPI cross-elevation / admin-only target / call from a background process or service) returns code:'ForegroundRestricted' ok:false rather than landing the click on the wrong window — recover by switching to a tool that accepts windowTitle directly (click_element / desktop_act) — browser_* tools target by tabId/selector, not windowTitle. A window on another virtual desktop returns WindowOnOtherDesktop and nothing is sent — the user's desktop is not switched. MouseClickNotDelivered is reserved-only (false-positive risk is too high to emit a typed code), so degradation is expressed via the 'unverifiable' status, not a separate error.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | X coordinate. Screen-absolute by default. When 'origin' is provided, treated as image-local (pixel position within the screenshot). | |
| y | Yes | Y coordinate. Screen-absolute by default. When 'origin' is provided, treated as image-local. | |
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues. | |
| fixId | No | One-shot fix approval ID. If a previous mouse_click returned a suggestedFix, pass that fixId here to approve it. The server revalidates the fix and executes with corrected args. fixId expires in 15 seconds and can only be used once. | |
| scale | No | Scale factor from screenshot response (only when dotByDotMaxDimension caused a resize). Omit if the screenshot was 1:1. Only used when 'origin' is also provided. | |
| speed | No | Cursor movement speed in px/sec. 0 = instant (teleport, no animation). Omit to use the configured default (DESKTOP_TOUCH_MOUSE_SPEED env var, default 3000). | |
| button | No | Mouse button to click | left |
| homing | No | Enable homing correction (default true). When enabled, the MCP server corrects stale coordinates if the target window moved since the last screenshot. Set false to disable all correction (like traction control OFF). | |
| lensId | No | Optional perception lens ID for advanced pinned-target workflows. When provided, guards are evaluated before clicking (safe.clickCoordinates, target.identityStable) and a perception envelope is attached to post.perception in the response. For normal use, omit lensId and pass windowTitle directly — Auto Perception handles tracking. | |
| origin | No | When set, (x,y) are image-local coords from a screenshot. Server converts to screen coords: screen_x = origin.x + x / (scale ?? 1), screen_y = origin.y + y / (scale ?? 1). Copy origin values directly from the screenshot response text. This eliminates manual coord math and prevents out-of-window clicks. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. "rich": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — "@active" and the dialog rescue both resolve one ("ambiguous_title"); or the target moved between the snapshot and the action ("target_changed"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see ("fix_target_unknown") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too ("tree_truncated"). The action itself is unaffected in every case; only the diff is. Default: "minimal". | minimal |
| settleMs | No | Milliseconds to wait after the action before checking foreground window (default 300). Only used when trackFocus=true. | |
| elementId | No | Hint: automationId of the UI element (from actionable[].id). Requires windowTitle. Used with elementName for more precise UIA re-query. | |
| forceFocus | No | When true, bypass Windows foreground-stealing protection via AttachThreadInput before focusing the target window. Required when a pinned window (e.g. Claude CLI) keeps stealing focus. Default: follows env DESKTOP_TOUCH_FORCE_FOCUS (default false). Set DESKTOP_TOUCH_FORCE_FOCUS=1 to make true the global default. | |
| trackFocus | No | When true (default), detect if focus was stolen from the target window after the action. Reports focusLost:{afterMs,expected,stolenBy,stolenByProcessName} in the response. Set false to skip the settle wait and focus check. | |
| doubleClick | No | Whether to double-click | |
| elementName | No | Hint: name/label of the UI element (from actionable[].name in screenshot(detail='text')). Requires windowTitle. Triggers UIA re-query to get fresh coordinates when the window resized or moved far. | |
| tripleClick | No | Whether to triple-click (select a line of text). Takes precedence over doubleClick when both are true. | |
| windowTitle | No | Hint: partial title of the window being clicked. Enables window-delta correction and auto-focus if the window went behind another. Use '@active' for the current foreground window. Example: "メモ帳", "Google Chrome" | |
| verifyDelivery | No | When true (default), capture pre/post snapshots of element-under-cursor + focusedElement + foregroundWindow + scrollPos to populate hints.verifyDelivery with status='delivered' | 'focus_only' | 'unverifiable' (issue #178). Set false to skip the extra UIA work when the caller will read post state itself. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so richly: auto-guarding semantics, post.perception.status values, hints.verifyDelivery status derivation from three signals, ForegroundRestricted recovery path, WindowOnOtherDesktop non-action, and the deliberate suppression of a MouseClickNotDelivered typed code. This is far more behavioral context than any structured field provides.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and the primary guidance, and nearly every sentence carries information (errors, recovery, verification). However it is a single dense block with many interleaved caveats, which taxes scanability for a click tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 21-parameter, enum-heavy tool with nested objects and no output schema, the description covers error codes, recovery alternatives, verification signalling, and guard behavior. An agent has everything needed to call it correctly and interpret failures.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds cross-parameter meaning the schema states only in isolation: origin+scale are meaningful ONLY with dotByDot=true responses and are converted to screen coords before guarding, and fixId is a one-shot 15s approval token. It adds real semantics beyond per-parameter docs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ("Click at screen coordinates") and immediately distinguishes itself from siblings by naming click_element (UIA native apps) and browser_click (Chrome) as preferred alternatives in their domains. An agent can place this tool vs. its neighbors without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use rules: normally pass windowTitle so the click is guarded, omit lensId for normal use, use doubleClick/tripleClick flags for multi-clicks. It even names the exclusions (prefer click_element for native apps, browser_click for Chrome) and gives concrete guarded vs. unguarded examples.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
mouse_dragA
Click and drag from (startX, startY) to (endX, endY) holding the left mouse button — for sliders, drag-and-drop, canvas drawing, and window resizing. Pass windowTitle so the server auto-guards the start coordinate and returns post.perception. Examples: mouse_drag({windowTitle:'Notepad', startX:50, startY:50, endX:200, endY:200}). lensId is optional and only for advanced pinned-target workflows. Caveats: Left button only. Both start and endpoint are guarded. Cross-window and desktop drags are blocked by default — pass allowCrossWindowDrag:true to confirm intent; that refusal is code:'CrossWindowDragBlocked'. A drag starting in a tabbed application's tab strip returns code:'TabDragBlocked' — pass allowTabDrag:true when tearing off or rearranging a tab is intended. hints.verifyDelivery:{status:'delivered'|'focus_only'|'unverifiable', reason} reports the post-drop observation in the same 3-value shape as mouse_click. MouseDragNotDelivered is SUGGESTS-registered but reserved-only (not emitted) — degradation is expressed via the 'unverifiable' status rather than a typed code. Win11 foreground refusal (UIPI cross-elevation / admin-only target / call from a background process or service) returns code:'ForegroundRestricted' ok:false from the homing path. A window on another virtual desktop returns WindowOnOtherDesktop and nothing is sent — the user's desktop is not switched.
| Name | Required | Description | Default |
|---|---|---|---|
| endX | Yes | ||
| endY | Yes | ||
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues. | |
| speed | No | Cursor movement speed in px/sec. 0 = instant (teleport, no animation). Omit to use the configured default (DESKTOP_TOUCH_MOUSE_SPEED env var, default 3000). | |
| homing | No | Enable homing correction (default true). When enabled, the MCP server corrects stale coordinates if the target window moved since the last screenshot. Set false to disable all correction (like traction control OFF). | |
| lensId | No | Optional perception lens ID. Guards and envelope same as mouse_click. | |
| startX | Yes | ||
| startY | Yes | ||
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| narrate | No | Narration level. "rich": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — "@active" and the dialog rescue both resolve one ("ambiguous_title"); or the target moved between the snapshot and the action ("target_changed"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see ("fix_target_unknown") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too ("tree_truncated"). The action itself is unaffected in every case; only the diff is. Default: "minimal". | minimal |
| windowTitle | No | Hint: partial title of the window being clicked. Enables window-delta correction and auto-focus if the window went behind another. Use '@active' for the current foreground window. Example: "メモ帳", "Google Chrome" | |
| allowTabDrag | No | When true, allow drags that start in the title-bar / tab-strip area of a tabbed app (Notepad, Terminal, Edge, Chrome, etc.). Default false — such drags are blocked because they detach the tab into a new window rather than moving the window. Pass true only when you intentionally want to rearrange or detach a tab. Note: active only when auto-guard is enabled (same scope as allowCrossWindowDrag). | |
| verifyDelivery | No | When true (default), capture pre/post snapshots of element-under-cursor + focusedElement + foregroundWindow + scrollPos to populate hints.verifyDelivery with status='delivered' | 'focus_only' | 'unverifiable' (issue #178). Set false to skip the extra UIA work when the caller will read post state itself. | |
| allowCrossWindowDrag | No | When true, allow dragging the endpoint into a different window or the desktop background. Default false — cross-window drags (including desktop/wallpaper) are blocked to prevent accidents. Pass true to confirm intent for deliberate cross-window or desktop-area drags. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so thoroughly: left-button-only, both endpoints guarded, cross-window/desktop blocked by default, tab-strip drags blocked, plus specific error codes (CrossWindowDragBlocked, TabDragBlocked, ForegroundRestricted, WindowOnOtherDesktop) and the verifyDelivery status semantics. It also discloses that a window on another virtual desktop sends nothing rather than switching desktops.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action, endpoint semantics, and use cases before diving into caveats, which is the right order. It is dense but most sentences earn their place; the niche 'MouseDragNotDelivered is SUGGESTS-registered but reserved-only' aside is the main place where trimming would help.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description must carry return semantics, and it covers them: post.perception, hints.verifyDelivery statuses, and the error-code taxonomy for blocked or refused drags. It is nearly complete for a 14-parameter mutation tool, with only minor gaps around the untouched params.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 71%, and the description adds genuine meaning beyond the schema for several parameters: windowTitle 'auto-guards the start coordinate and returns post.perception', lensId is scoped as advanced-only, and the two guard flags are framed by intent. It leaves some params (speed, homing, include, narrate, hwnd) entirely to the schema, so it does not fully compensate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource with both endpoints ('Click and drag from (startX, startY) to (endX, endY)') and enumerates concrete use cases (sliders, drag-and-drop, canvas drawing, window resizing). It implicitly and explicitly distinguishes itself from the sibling mouse_click by referencing the shared verifyDelivery shape and by its distinct drag semantics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear when-to-use context via the use-case list and a worked example, plus explicit conditions for the advanced flags (pass allowCrossWindowDrag:true to confirm cross-window intent; allowTabDrag:true for tearing off tabs). It does not, however, explicitly route the agent to mouse_click or scroll as alternatives, so it stops short of full when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
notification_showA
Show a Windows system tray balloon notification to alert the user. Use at the end of a long-running task so the user knows it finished without watching the screen. Caveats: toast の user reach は原理的に観測不能 (matrix §3.1 line 158 規範整合)。Focus Assist (Do Not Disturb) / Notifications-off setting / consent UI sink いずれも tool 側からは判別不能のため、successful response は常に hints.verifyDelivery を含む (status="unverifiable", reason="user_visible_side_effect_uninspectable", channel="win32_balloon_tip" — 全 double-quoted JSON literal)。caller は user 側の post-notification behavior (例: wait_until(focus_changes)) で間接観測することが望ましい。Uses System.Windows.Forms — no external modules needed.
| Name | Required | Description | Default |
|---|---|---|---|
| body | Yes | Notification body text | |
| title | Yes | Notification title | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations present, the description carries the full burden and does so richly: it discloses that user-visible delivery is fundamentally unobservable, that Focus Assist / notifications-off / consent-UI suppression cannot be detected from the tool, and that every successful response includes hints.verifyDelivery with status="unverifiable", reason, and channel. This pre-empts the exact behavioral trap an agent would otherwise fall into.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and usage are correctly front-loaded, but the caveat block mixes English with Japanese and cites opaque internal references ("matrix §3.1 line 158 規範整合"), which slows an English-language agent parsing the key constraint. The double-quoted JSON literal detail is useful but the surrounding prose is dense and non-uniform in language.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema or annotations exist, and the description compensates by specifying the shape of the delivery-verification hint on success. Parameters are fully covered by the schema. It stops short of 5 because the include/envelope return-shape options are left entirely to the schema and no failure-mode behavior is described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so title, body, and include are already documented with constraints (maxLength 200/500) and the envelope/raw opt-in semantics. The description adds no parameter-level meaning, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ("Show") and resource ("a Windows system tray balloon notification") with the mechanism named (System.Windows.Forms). An agent can identify the capability immediately, and no sibling tool overlaps with desktop notification delivery.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a concrete trigger: "Use at the end of a long-running task so the user knows it finished without watching the screen." It also advises the caller to indirectly observe delivery via wait_until(focus_changes), which is actionable routing. It lacks an explicit when-not, so it falls short of 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
run_macroA
Purpose: Execute multiple tools sequentially in one MCP call — eliminates round-trip latency for predictable multi-step workflows.
Details: steps[] is an array of {tool, params} objects. Not every tool is a valid step: excel, key_locker, server_status, screenshot_query, screenshot_gc and run_macro itself never are, and whether desktop_discover/desktop_act or get_windows/get_ui_elements/set_element_value are depends on this server's configuration — each pair is refused in the other one. Everything else in the tool list is a step. The tool field's own description names the exact set THIS server dispatches. Plus a special sleep pseudo-step: {tool:"sleep", params:{ms:N}} (max 10000ms per step). stop_on_error=true (default) halts on first failure. Max 50 steps. The LLM cannot inspect intermediate results during execution — all steps run to completion (or first error) before any output is returned.
Prefer: Use for predictable fixed sequences (focus → sleep → type → screenshot). Do not use for conditional logic — return to the LLM between branches so it can inspect intermediate state.
Caveats: If any step may fail conditionally (e.g. a dialog that may or may not appear), split the macro at that point. Each screenshot step within a macro incurs the same token cost as a standalone call.
Examples:
[{tool:'focus_window',params:{windowTitle:'Notepad'}},{tool:'sleep',params:{ms:300}},{tool:'keyboard',params:{action:'type',text:'Hello'}},{tool:'screenshot',params:{detail:'text',windowTitle:'Notepad'}}]
[{tool:'browser_navigate',params:{url:'https://example.com'}},{tool:'wait_until',params:{condition:'element_matches',target:{by:'text',pattern:'Example Domain'}}}]
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | Ordered list of tool calls to execute sequentially (max 50 steps). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| stop_on_error | No | Stop execution on the first error (default true). Set false to collect all results. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden and does so well: it discloses the max 50 steps, stop_on_error default, 10000ms sleep cap, that intermediate results cannot be inspected mid-run, the token cost of embedded screenshots, and which sibling tools are never valid steps. It does not describe the success/failure return shape (e.g. per-step error reporting), which is the main remaining gap for a tool with no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Long but well-front-loaded with Purpose/Details/Prefer/Caveats/Examples labels, and the two worked examples remove ambiguity. There is minor redundancy — the config-dependent tool pairs are explained twice — and the volume exceeds what most agents need, but almost every sentence carries specific operational guidance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex batch-execution tool with no annotations and no output schema, the description covers all decision-critical constraints an agent needs: valid step set, step limits, error-handling behavior, latency/cost implications, and when to avoid it. Nothing required to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning beyond the schema: the special 'sleep' pseudo-step with its max duration, the config-dependent validity of desktop_discover/desktop_act vs get_windows/get_ui_elements/set_element_value, and the note that the tool field's own description is authoritative for the dispatched set. The include parameter is left to the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Execute multiple tools sequentially in one MCP call') and immediately names the benefit and its distinguishing scope versus any single sibling tool. An agent can tell this is the macro/batch aggregator, not one of the ~30 individual action tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit 'Prefer' section names the use case (predictable fixed sequences like focus → sleep → type → screenshot) and the explicit exclusion (do not use for conditional logic — return to the LLM between branches). It also advises splitting the macro where a step may fail conditionally, giving a concrete when-not rule.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotA
Purpose: Capture desktop, window, or region across detail levels (meta / text / image / som / ocr) and capture modes (normal / background). Details: detail='meta' (default) returns window titles+positions only (~20 tok/window, no image). detail='text' returns UIA actionable elements with clickAt coords, no image (~100-300 tok). detail='som' returns OCR-detected elements with IDs plus a Set-of-Marks annotated image delivered by-ref by default (bypasses UIA entirely). detail='ocr' returns Windows OCR words with screen-pixel clickAt coords (Phase 4: absorbs former screenshot_ocr — use when UIA is sparse and you want to force OCR unconditionally). detail='image' and detail='som' both return a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to also embed the inline image (the annotated bitmap for som). mode='background' captures the window itself, hidden/minimised/occluded too — from its composition surface (WGC) when it can, else via PrintWindow (Phase 4: absorbs former screenshot_background) — pair with windowTitle/hwnd. dotByDot=true returns 1:1 pixel WebP; compute screen coords: screen_x = origin_x + image_x (or screen_x = origin_x + image_x / scale when dotByDotMaxDimension is set — scale printed in response); a WGC frame's origin is the window's visible frame, and when that cannot be read it prints none. diffMode=true returns only changed windows after the first call (~160 tok). region={x,y,width,height} captures a sub-rectangle (Phase 4: absorbs former scope_element when paired with windowTitle/hwnd — discover element bounds via desktop_discover, then pass region here). Data reduction: grayscale=true (−50%), dotByDotMaxDimension=1280 (caps longest edge), windowTitle+region (sub-crop to exclude browser chrome — e.g. region={x:0, y:120, width:1920, height:900}). Prefer: Use meta to orient, text before clicking, dotByDot only when precise pixel coords are needed. Use detail='som' for native apps or games that do not expose UIA elements (UIA-Blind). Use detail='ocr' for OCR-only (skip UIA entirely). Use mode='background' when the target window must stay hidden or cannot be brought to foreground. Prefer browser_* tools for Chrome. Use diffMode after actions to confirm state changed. Only use image+confirmImage when text returned 0 actionable elements and visual inspection is genuinely required. Caveats: Default mode scales to maxDimension=768 — image pixels ≠ screen pixels; apply the scale formula before passing to mouse_click. Foreground detail='image' returns a by-ref resource_link by default; pass confirmImage=true to also receive inline pixels. diffMode requires a prior full-capture baseline (non-diff call or workspace_snapshot) — calling diffMode cold returns a full frame, not a diff. mode='background' requires windowTitle or hwnd, and only composes with detail in {'image','meta'} — detail='text'/'som'/'ocr' run only against foreground capture (the dispatcher rejects the conflicting combination). Passing mode='background' is itself the acknowledgement that image pixels are wanted, so confirmImage is NOT required for it (matches the former screenshot_background contract). fullContent=false enables legacy mode (faster but GPU windows may be black). detail='ocr' requires windowTitle or hwnd; first call may take ~1s (WinRT cold-start) and the matching OCR language pack must be installed. Examples: screenshot() → meta orientation of all windows screenshot({detail:'text', windowTitle:'Notepad'}) → clickable elements with coords screenshot({detail:'ocr', windowTitle:'PDF', ocrLanguage:'ja'}) → OCR words with screen-pixel coords screenshot({mode:'background', windowTitle:'Chrome', dotByDot:true, dotByDotMaxDimension:1280, grayscale:true}) → background-capture pixel-accurate Chrome screenshot({windowTitle:'Notepad', region:{x:0,y:120,width:600,height:400}}) → cropped sub-region (zoom into element after desktop_discover)
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). Obtain from desktop_discover (windows[].hwnd). String type to avoid 64-bit precision issues. | |
| mode | No | Capture mode. 'normal' — default. Window-targeted captures (windowTitle / hwnd) use Win32 PrintWindow; when it returns no data or an all-black frame, the window's composition surface (WGC) is tried next if the window is visible on the current desktop (not minimised or cloaked), and BitBlt last. The route used is reported in hints.captureSource ('printwindow' | 'wgc' | 'bitblt-fallback'). Fullscreen / displayId captures use BitBlt. 'background' — a capture of the window itself, not of the screen: its composition surface (WGC) when it can be for a window visible on the current desktop (not minimised or cloaked) with fullContent, else Win32 PrintWindow. Requires windowTitle (or hwnd). Pair with fullContent for GPU-rendered apps. | normal |
| detail | No | Response detail level (omit to let the server pick a smart default): omitted — auto: 'image' when dotByDot/region/displayId is specified, else 'meta' 'meta' — window title + screen region only (~20 tok/window, cheapest) 'text' — UIA element tree as JSON with text values (~100-300 tok/window, no image) 'image' — actual screenshot pixels. Returns a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to ALSO embed the inline image. 'som' — Set-of-Marks elements + annotated image (bypasses UIA entirely). Returns the OCR elements[] plus a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to ALSO embed the annotated bitmap. 'ocr' — Windows OCR words with screen-pixel clickAt coords (Phase 4: absorbs former screenshot_ocr). Use when UIA returns no actionable elements (WinUI3 custom-drawn UIs, game overlays, PDF viewers). Note: detail='text' auto-falls back to OCR via ocrFallback='auto'; choose detail='ocr' only when forcing OCR unconditionally. | |
| region | No | Capture only this sub-region. Without windowTitle: virtual screen coordinates. With windowTitle: window-local coordinates — useful to exclude browser chrome (tabs/address bar). Example: windowTitle='Chrome', region={x:0, y:120, width:1920, height:900} skips the 120px browser chrome. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| diffMode | No | Layer diff mode — compares each window against the buffered previous frame. First call = full I-frame (all windows). Subsequent calls = only changed windows (P-frame). Implicitly enables dotByDot. Best used with windowTitle=undefined to snapshot all windows. | |
| dotByDot | No | 1:1 pixel mode — no scaling, WebP compression. Window captures include 'origin: (x,y)' so you can compute screen position: screen_x = origin_x + image_x. A frame from the window's composition surface (WGC: mode='background' with fullContent on a window visible on the current desktop — not minimised or cloaked — when it can be, or normal mode's rescue) starts at the window's visible frame, and its origin is that frame's corner; when the visible frame cannot be read, no origin is given — the text says so; use desktop_discover for coordinates. When dotByDotMaxDimension is also set, scale factor is included: screen_x = origin_x + image_x / scale. | |
| displayId | No | Capture a specific monitor (0 = primary). Use desktop_state({includeScreen:true}) to list displays. | |
| grayscale | No | Convert to grayscale before encoding. Reduces file size ~50% for text-heavy content (e.g. AWS console, code editors). Avoid when color is meaningful (charts, status indicators). | |
| fullContent | No | When mode='background', use PW_RENDERFULLCONTENT to capture GPU-rendered windows (Chrome, Electron, WinUI3). Default true. Set false for legacy mode (faster but GPU windows may appear black). Ignored unless mode='background'. | |
| ocrFallback | No | OCR fallback behaviour when detail='text'. 'auto' (default): fire Windows OCR if UIA returns 0 actionable elements OR hints.uiaSparse=true (UIA returned <5 elements, typical for Chrome). 'always': always augment actionable[] with OCR words. 'never': disable OCR entirely. | auto |
| ocrLanguage | No | BCP-47 language tag for the OCR engine (e.g. 'ja', 'en-US'). Auto-detects from system locale when omitted. Used when detail='text' (OCR fallback) or detail='ocr' (direct OCR). | |
| webpQuality | No | WebP quality when dotByDot=true or diffMode=true. 40=layout only, 60=general (default), 80=fine text. | |
| windowTitle | No | Capture only the window whose title contains this string. Use '@active' for the current foreground window. Prefer over full-screen when target window is known. | |
| confirmImage | No | Embed inline image pixels in the response. detail='image' now returns a cheap by-ref resource_link WITHOUT this flag (it is no longer blocked); confirmImage=true ADDITIONALLY embeds the inline image for immediate vision. detail='som' likewise returns its elements[] + a by-ref resource_link by default; confirmImage=true ADDITIONALLY inlines the annotated SoM bitmap. Prefer detail='text' / diffMode=true / dotByDot=true first — set confirmImage=true only when inline visual inspection is genuinely required. | |
| maxDimension | No | Max width or height in pixels (default 768). Use 1280 to read small text, code, or fine UI details. Ignored when dotByDot=true. | |
| preprocessPolicy | No | OCR preprocessing scale policy for detail='som' and OCR fallback paths. 'auto' (default): clamp scale to 1 on OOM (>8MP) or high-DPI (≥150%). 'aggressive': relaxes DPI clamp to 175%, preserving upscale on 150%-DPI monitors (e.g. Outlook PWA). Also auto-enables adaptive binarization. 'minimal': always scale=1 regardless of DPI/resolution. | auto |
| preprocessAdaptive | No | When true, apply Sauvola adaptive binarization after contrast stretch. Improves recognition of thin text on low-contrast or gradient backgrounds. Automatically enabled when preprocessPolicy='aggressive'. Requires Rust native engine; silently skipped otherwise. | |
| dotByDotMaxDimension | No | Cap the longest edge (pixels) when dotByDot=true. Reduces payload while preserving coordinate math. Example: 1280 on a 1920×1080 screen → scale≈0.667. Response includes scale factor: screen_x = origin_x + image_x / scale. Recommended for Chrome: dotByDot=true, dotByDotMaxDimension=1280, grayscale=true. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full disclosure burden, and it does so extensively: scaling formula caveat (maxDimension=768 means pixels != screen pixels), diffMode requiring a prior baseline or returning a full frame cold, background mode requiring windowTitle/hwnd and only composing with {image, meta}, ocr cold-start ~1s plus language pack requirement, and confirmImage not needed for background mode. This is well beyond what annotations would supply.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is long, but that length is justified by 19 parameters and five detail modes. The Purpose/Details/Prefer/Caveats/Examples structure is front-loaded and scannable, and each section earns its place. There is minor duplication with the already-exhaustive schema descriptions, keeping it just short of a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description must explain return shapes, and it does: meta returns titles+positions, text returns UIA elements with clickAt, som/ocr return elements plus by-ref resource_link, and confirmImage controls inlining. Combined with the caveat coverage of mode/detail compatibility, an agent has enough to call this complex tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents each parameter, making 3 the baseline. The description nevertheless adds cross-parameter meaning the schema does not: worked examples, the grayscale/dotByDotMaxDimension/windowTitle+region reduction recipes, and the screen-coordinate computation with origin and scale. That is genuine value over the per-parameter text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence names a specific verb (Capture) and its resource scope (desktop, window, region) and enumerates the orthogonal axes it operates on (detail levels, capture modes). It separates itself from siblings directly by routing Chrome work to browser_* and element discovery to desktop_discover. An agent can tell what this tool does and does not do without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer:' block gives explicit when-to-use rules: meta to orient, text before clicking, som for UIA-blind native apps/games, ocr to force OCR unconditionally, background when the window must stay hidden, diffMode after actions to confirm state, and browser_* for Chrome. Exclusions and alternatives are stated rather than implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshot_gcA
Reclaim disk space from cached screenshots by retention policy. By DEFAULT this is a dry run: it returns the captures that WOULD be deleted (candidates) plus a count/size of leftover orphan files, and deletes nothing. To actually delete, pass BOTH dryRun:false AND confirm:true. Retention caps (all optional): maxCount (keep newest N), maxTotalBytes (keep newest under a byte budget), maxAgeMs (delete older than). When you pass none, the cache's env defaults apply (newest 200 / 256 MiB). Scope to a single tag with tag (other tags are never touched); includeOrphans (default true) also reclaims leftover on-disk files with no index entry. The newest capture is always kept by the count/byte caps. Only ever touches files inside the screenshot cache — never any other path.
| Name | Required | Description | Default |
|---|---|---|---|
| tag | No | Limit deletion to captures under this tag (case-insensitive). Other tags are never touched. | |
| dryRun | No | Default true: only LIST what would be deleted, delete nothing. Set false (with confirm:true) to actually delete. | |
| confirm | No | Safety gate: deletion happens ONLY when dryRun:false AND confirm:true. Otherwise the call is forced to a dry run. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| maxAgeMs | No | Delete captures older than this many milliseconds (opt-in; can clear even the newest). | |
| maxCount | No | Keep only the newest N captures; delete the rest. The single newest is always kept. | |
| maxTotalBytes | No | Keep the newest captures under this total byte budget; delete older ones beyond it. The newest is always kept. | |
| includeOrphans | No | Default true: also reclaim leftover on-disk image files that are not tracked in the cache index (e.g. files left behind by a crash). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations supplied, the description carries the full behavioral burden and discharges it: it discloses the destructive safety gate (dual-flag), what a dry run returns (candidate captures plus orphan count/size), that includeOrphans reclaims files with no index entry, that the newest capture survives count/byte caps, and the hard scope guarantee that only files inside the screenshot cache are ever touched.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but well front-loaded: the default-safe dry run and the deletion gate come first, then retention caps, then scoping. It repeats the dryRun/confirm pairing and re-explains orphan reclaiming that the schema also covers, so a little trimming is possible.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 8-parameter, annotation-free mutation tool with no output schema, this is close to complete: deletion semantics, safety gating, defaults, scoping, and even the dry-run return shape are all covered. Only permission/auth requirements are left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 100%, so the baseline is 3, but the description adds genuine meaning beyond the schema: the concrete env defaults (newest 200 / 256 MiB) applied when no caps are given, the interplay of dryRun and confirm, and that maxAgeMs can clear even the newest capture. It does not explain the `include` response-shape parameter at all.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb and resource: 'Reclaim disk space from cached screenshots by retention policy.' It is unmistakably distinct from the read-oriented siblings (screenshot, screenshot_query) and the browser_* family, and the retention-policy scope further narrows what it does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when/when-not rules: the call is a dry run by DEFAULT and deletes nothing, and real deletion requires BOTH dryRun:false AND confirm:true. It also states the fallback behavior when no retention caps are passed (env defaults of newest 200 / 256 MiB) and the tag-scoping condition.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshot_queryA
List screenshots already saved in the disk-cache WITHOUT re-reading any pixels. The screenshot tools return each capture as a cheap by-ref link (screenshot://by-ref/{captureId}); this lists what is in the cache — captureId + by-ref uri, dimensions, size in bytes, timestamp, and tag/window — so you can find and re-open a specific earlier capture, or check how much the cache holds before reclaiming space with screenshot_gc. The response also carries whole-cache totals (totalCaptures / totalBytes). Reading a capture's bytes still costs tokens, so open a by-ref link only when you actually need to inspect the pixels. Filter by tag (case-insensitive) / windowUuid / since / until; page with limit (default 50) and offset. Results are newest-first and never include a filesystem path.
| Name | Required | Description | Default |
|---|---|---|---|
| tag | No | Filter to captures stored under this tag (case-insensitive). Omit to list all. | |
| limit | No | Maximum rows to return, newest first (default 50, max 500). | |
| since | No | Only captures taken at/after this time (epoch milliseconds, inclusive). | |
| until | No | Only captures taken at/before this time (epoch milliseconds, inclusive). | |
| offset | No | Rows to skip from the newest end, for paging (default 0). | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| windowUuid | No | Filter to captures of a specific window (the window's stable id). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and delivers: the by-ref link return convention (screenshot://by-ref/{captureId}), the fact that reading bytes costs tokens, newest-first ordering, whole-cache totals in the response, and that results never include a filesystem path. It does not cover auth/rate limits, but the behavioral profile is unusually complete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose, then the response shape, then filtering/paging, then the ordering guarantee. Dense with many parentheticals, but each sentence carries information an agent needs. Slightly overpacked but not wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description compensates by enumerating the returned fields (captureId + by-ref uri, dimensions, size in bytes, timestamp, tag/window) and the totals (totalCaptures/totalBytes), plus ordering. Nothing an agent needs to call this correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds real meaning beyond the schema: it groups the filters (tag/windowUuid/since/until), states the paging defaults (limit default 50, offset), and clarifies that results are newest-first and offset skips 'from the newest end', which disambiguates the offset direction the schema leaves implicit.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List screenshots already saved in the disk-cache') and sharpens it with a distinguishing constraint ('WITHOUT re-reading any pixels'). It also implicitly separates this from sibling `screenshot` (which captures) and names `screenshot_gc` as the space-reclaim alternative, so an agent can tell the tools apart without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives the two motivating use cases ('find and re-open a specific earlier capture, or check how much the cache holds before reclaiming space with screenshot_gc') and names the alternative tool for space reclamation. It also adds an exclusion-style warning: open a by-ref link only when you actually need the pixels.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scrollA
Purpose: Scroll a window or page. 5 strategies via action: 'raw' (wheel notches), 'to_element' (UIA name/automationId or CSS selector), 'smart' (auto-detect target with multi-strategy fallback), 'capture' (full-page stitched image), 'read' (scroll+OCR+dedupe → stitched text). Details: action='raw': send raw mouse-wheel notches at (x,y) or current cursor, optional window focus. Scroll scale — one amount unit is one wheel notch on every dispatch path, and ≈10 notches move one screenful: UIA-capable apps (Notepad, Explorer, WPF) step ≈1/10 of the visible area per notch, browsers and WebView apps (Chrome, Electron, Tauri) ≈100 px. amount:3 (default) ≈ a third of a screen. action='to_element': scroll a named element into viewport (UIA or CDP). action='smart': handles nested scroll layers, virtualised lists, sticky-header occlusion. action='capture': stitches full-page images (caps at ~700KB raw); sizeReduced=true means downscaled. action='read': scrolls page-by-page, OCRs each viewport, deduplicates overlapping lines, returns stitched text; language auto-detected from OS locale if omitted. Prefer: Use action='to_element' or action='smart' for click target out-of-viewport recovery (entity_outside_viewport) — scrolling only helps when the target scrolled out of its own window; if desktop_act reported origin_window_not_visible, restore the window with focus_window and re-run desktop_discover instead. Use action='capture' for reading long pages as images. Use action='read' for extracting text from long native-app documents (PDF readers, text editors, terminals) where copy-paste is unavailable. For simple scroll without target, use action='raw'. Caveats: action='capture' returns stitched image — pixels do NOT match screen coords when sizeReduced=true, use for reading only, not mouse_click. action='smart' CDP path requires browser_open. action='to_element' native path requires element to implement UIA ScrollItemPattern. action='read' uses OCR (imperfect accuracy) and requires the window to be visible; for browser pages prefer browser_eval or browser_overview for accurate DOM text. action='raw' typed errors: code:'ScrollNotDelivered' on silent drop (overlay / non-scrollable / UIPI low-IL); already-at-boundary is success via pre/post-percent disambiguation. hints.verifyDelivery.{channel,reason} per ADR-018 §2.6 — channel is the transport used ('uia'/'cdp'/'postmessage'/'wheel_send_input'); reason='pixel_delta_observed' = pixel evidence only (weak). action='smart' typed errors: code:'OverflowHiddenAncestor' (retry with expandHidden:true), code:'VirtualScrollExhausted' (provide virtualIndex). Examples: scroll({action:'raw', direction:'down', amount:5, windowTitle:'Chrome'}) scroll({action:'to_element', name:'OK', windowTitle:'Dialog'}) scroll({action:'smart', target:'#create-release-btn'}) scroll({action:'capture', windowTitle:'Chrome', maxScrolls:10}) scroll({action:'read', windowTitle:'Acrobat', maxPages:15}) // OCR + dedupe long PDF
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | X coordinate to scroll at (moves cursor there first) | |
| y | No | Y coordinate to scroll at | |
| hint | No | Scroll direction hint for binary-search (image path). Seeds lo/hi bounds to reduce attempts. | |
| hwnd | No | Direct window handle ID (takes precedence over windowTitle). | |
| name | No | Partial name/label of the element (UIA name match). Use for native app elements. At least one of name or selector must be provided. | |
| port | No | 'to_element': CDP port for Chrome path (default 9222) 'smart': CDP port (default 9222) | |
| block | No | Vertical alignment after scroll — start/center/end/nearest (Chrome path only, default: center) | center |
| speed | No | Cursor movement speed in px/sec (0=teleport, omit=default) | |
| tabId | No | 'to_element': Tab ID (Chrome path only). Omit for first page tab. 'smart': CDP tab ID (Chrome path only). Omit for first page tab. | |
| action | Yes | Action selector — one of: raw, to_element, smart, capture, read. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| amount | No | Number of scroll notches (default 3, max 1000). One notch = one physical wheel detent on every dispatch path. Count screenfuls, not lines: ≈10 notches move one screenful, so amount:3 (default) is about a third of a screen. UIA-capable apps (Notepad, Explorer, WPF) step ≈1/10 of the window's visible area per notch; browsers and WebView-based apps (Chrome, Electron, Tauri) move ≈100 px per notch. Exact distance depends on the app, the window size and the OS wheel-speed setting. The 1000-notch ceiling exists because each notch is dispatched as real wheel input; to reach a specific place in a long document use action='to_element' or action='smart' instead of a huge amount. | |
| homing | No | Apply window-movement homing correction to (x,y) before scrolling. Default true. | |
| inline | No | Vertical alignment after scroll (CDP path). Default: center. | center |
| target | No | CSS selector (Chrome/Edge) or partial UIA name (native apps). For CDP path, must be a valid CSS selector (starts with #, ., tag, or [ ). For UIA path, a partial name match against element Name property. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| language | No | OCR language code (e.g. 'ja', 'en', 'zh'). Omit to auto-detect from Windows system locale via Intl.DateTimeFormat().resolvedOptions().locale. Default: auto. | |
| maxDepth | No | Max number of ancestor scroll containers to walk. Default 3. | |
| maxPages | No | Maximum number of scroll steps / OCR pages (default 20, max 50). | |
| maxWidth | No | Max size of the short edge of the final image (default 1280). For 'down': caps the image width; height is unconstrained. For 'right': caps the image height; width is unconstrained. | |
| selector | No | CSS selector for the element (Chrome/Edge only). At least one of name or selector must be provided. | |
| strategy | No | auto (default): try CDP → UIA → image in order. cdp: Chrome/Edge only. uia: native Windows UIA. image: image + Win32 binary-search. | auto |
| direction | No | 'raw': Scroll direction 'smart': Scroll direction. into-view: scroll until target element is visible (default). Other values scroll unconditionally. 'capture': Scroll direction: 'down' (vertical, uses Page Down key) or 'right' (horizontal, uses mouse scroll). Default 'down'. | |
| scrollKey | No | Key sent to scroll one page. PageDown (default): full-page scroll for most apps. Space: web/PDF readers. ArrowDown: line-by-line slow scroll. | PageDown |
| maxScrolls | No | Maximum scroll iterations before stopping (default 10, max 30) | |
| retryCount | No | Max scroll attempts (image path binary-search). Default 3, cap 4. | |
| windowTitle | No | 'raw': Partial window title. When provided, the server focuses this window first. 'to_element': Partial window title (required for native path when name is used) 'smart': Partial window title. Required for UIA and image paths. For CDP path, optional. 'capture': Partial title of the window to capture (case-insensitive match) 'read': Partial window title to focus and OCR (case-insensitive match). | |
| expandHidden | No | Temporarily set overflow:hidden ancestors to overflow:auto to unlock scroll. Mutates live CSS. | |
| virtualIndex | No | Target row index in a virtualised list (0-based). Enables direct TanStack/data-index seeking. | |
| virtualTotal | No | Total row count in a virtualised list. Required when virtualIndex is set. | |
| scrollDelayMs | No | 'capture': Milliseconds to wait after each scroll for rendering to settle (default 400). Increase for slow/animated pages. 'read': Milliseconds to wait after each scroll for rendering to settle (default 400). | |
| verifyWithHash | No | Verify scroll effectiveness via perceptual hash comparison. Automatically enabled for image path. | |
| stopWhenNoChange | No | Stop automatically when two consecutive pages yield no new lines after deduplication (page-end detection). Default true. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and delivers strongly: it discloses typed errors, OCR imperfection, pixel-coordinate mismatch after downscaling, visibility/UIA/CDP prerequisites, boundary-success semantics, per-action caveats, and ADR-backed verification hints. This is unusually complete behavioral disclosure for a tool with zero structured safety information.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with Purpose, then Details, Prefer, Caveats, and Examples. Its length is justified by five actions and 32 parameters, and each section helps an agent select or correctly invoke the tool rather than padding the text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 32-parameter, five-action tool with no annotations and no output schema, the description supplies routing guidance, action behavior, error recovery, caveats, and examples that cover what an agent needs to call it correctly. Return-shape details are partial but sufficient given the action-specific outputs described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description adds meaningful action-level semantics and examples beyond the flat schema, especially by defining what each action does and implying per-action required fields. However, much parameter detail still lives in the schema, and some amount-scale language is duplicated rather than adding new meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Scroll a window or page') and immediately enumerates five distinct action strategies, making it easy to distinguish from sibling browser_* and desktop_* tools. The agent can tell what the tool does and which strategy family applies without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer:' block gives explicit routing: use to_element/smart for out-of-viewport click recovery, capture for reading long pages as images, read for OCR text extraction where copy-paste is unavailable, and raw for simple scrolling. It also names alternatives (focus_window, desktop_discover, browser_eval, browser_overview) and the conditions that select them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
server_statusA
Return MCP server status. engine: native engine availability — uia: 'native' = Rust UIA addon (~2 ms focus / ~100 ms tree); 'powershell' = PS fallback (~366 ms focus). nativeUia says why: 'native', 'disabled' (DESKTOP_TOUCH_DISABLE_NATIVE_UIA=1 takes the native UIA engine out and keeps the rest of the addon: UIA calls with a PowerShell version use it, the rest are skipped), or 'unavailable' (the addon has no UIA engine). nativeUiaEvidence says whether the native UIA engine actually ran in this process, as the engine and the OS answer rather than the switch: comThreadStarts and tasksSent (the engine's own counts of attempts), tasksDone (tasks the engine finished; far below tasksSent means native UIA is stuck and every UIA call is waiting out its timeout), focusRegistration ('pending' for long means focus events are off because a desktop-wide registration is not returning), uiaCoreLoaded (whether UIAutomationCore.dll is loaded, supporting evidence only); null = the addon cannot say. It changes per call, so read it after the act it should cover. imageDiff: 'native' = Rust SSE2 SIMD (0.26 ms @ 1080p); 'typescript' = TS fallback (~3.8 ms). health: process diagnostic snapshot (issue #365) — pid (which process these numbers are from; the evidence counts restart from zero in a new one), uptimeSec, memory.{rssBytes,heapUsedBytes,heapTotalBytes}, cpu.{userUs,systemUs} (cumulative since startup), shutdown.{pending,graceMs,inflightCount} (pending=true means stdin EOF received and grace timer is running), lastRpc.{receivedAt(ISO),method} (last JSON-RPC request observed on stdio transport; HTTP transport is not tracked). Diagnostic metadata — do not surface unless the user asks about performance/troubleshooting. engine values other than nativeUiaEvidence are stable for the process lifetime; nativeUiaEvidence and health change per call.
| Name | Required | Description | Default |
|---|---|---|---|
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does so substantially: it discloses that engine values are stable for the process lifetime while nativeUiaEvidence and health change per call, that health counters reset in a new process, and that the tool is a diagnostic read. It stops short of stating the safety profile (no side effects / auth) explicitly, keeping it from a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose is front-loaded in one short sentence, but the remaining ~300 words are an unstructured run-on dumping every internal field's semantics, including details the description itself says not to surface unless asked. Sized and structured poorly for tool selection, even if the individual facts are accurate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a high-complexity diagnostic tool with no output schema, the description exhaustively documents the returned fields (engine, nativeUia/nativeUiaEvidence, imageDiff, health) and their stability semantics, so an agent can interpret the response. The only real omission is the `include` response-shape option, which is covered in the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 100% description coverage for the single `include` parameter (envelope vs raw shape), so the schema already does the heavy lifting. The description never mentions `include` or the envelope/raw response-shape choice, adding nothing beyond the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence gives a specific verb+resource ('Return MCP server status') that an agent can act on immediately. It is clearly distinguishable from the action-oriented siblings (browser_*, click_element, etc.) by being diagnostic in nature, though the description never explicitly says so. The purpose is clear but buried under heavy field-level detail.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It offers scattered timing guidance ('read it after the act it should cover') and output guidance ('do not surface unless the user asks about performance/troubleshooting'), which is useful. However, it never states when to call server_status versus the sibling tools or what question it answers, so invocation guidance is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
terminalA
Purpose: Interact with a terminal window: read output, send input, or run+wait+read in one call. action='read' / action='send' absorb the formerly-standalone read/send tools (Phase 4).
Details: action='run' is the recommended high-level workflow: send command → wait until quiet/pattern/timeout → read output. The command text is passed as input (the legacy parameter name command is also accepted as a deprecated alias — see issue #245). Returns completion={reason, elapsedMs} first-class plus outputIntegrity:'ok'|'baseline_lost' so callers can detect when scrollback could not be anchored to the pre-send buffer. action='read' reads current text via UIA TextPattern (falls back to OCR); use sinceMarker for incremental diff. action='send' sends a command with focus management.
Prefer: action='run' for command execution + result. For long-running commands (test runners, builds, deploys) use until:{mode:'pattern', pattern:''} — the default quiet mode is tuned for short interactive commands and may complete prematurely on multi-second silent gaps mid-run. Use action='read'/'send' for fine-grained control or when you need to interleave other actions. read/send/run also accept a launch_console paneId (pass the paneId field, NOT windowTitle) to keep targeting a pane after its title changes.
Caveats: Do not screenshot the terminal — action='read' is cheaper and structured. action='run' supports completion reasons: quiet | pattern_matched | exited | timeout | window_closed | window_not_found | send_failed (send rejected on a live window — see warnings for the underlying error code). until:{mode:'exit', shell:'bash'|'powershell'} (issue #386) returns completion.exitCode + reason:'exited' via an echo-immune sentinel that works for multiline input that pattern mode cannot anchor; pass shell explicitly (auto fails as ExitModeShellAmbiguous on WT/conhost/SSH), cmd is unsupported (ExitModeShellUnsupported), open-construct input is rejected (ExitModeUnsafeInput). When outputIntegrity:'baseline_lost' is returned, output is forced to '' and readError.code='BaselineMarkerLost' is set: rerun with until:{mode:'pattern',...} or longer timeoutMs. action='run' may also emit warnings prefixed FileLockCollision: when output reveals an EBUSY/Windows-lock/EAGAIN-EDEADLK file collision (issue #236). Default quietMs=1500 (issue #196); long silences require pattern mode. preferClipboard=true (send default) replaces the clipboard. Hidden-input prompts emit verifyDelivery.unverifiable (reason:'hidden_input_prompt') — use method:'foreground'. action='read' typed errors: TerminalWindowNotFound, TerminalTextPatternUnavailable (force source:'ocr'); stale sinceMarker → hints.terminalMarker.previousMatched:false on ok:true (omit sinceMarker). FG-path Win11 foreground refusal returns code:'ForegroundRestricted' — switch to method:'background' or DTM_BG_AUTO=1. Off-desktop target → WindowOnOtherDesktop, not sent. BG path auto-engages only when the target class is ConsoleWindowClass (conhost: cmd / PowerShell / pwsh) OR env DTM_BG_AUTO=1. Windows Terminal (CASCADIA_HOSTING_WINDOW_CLASS) is EXCLUDED (issue #173): WT runs on WinUI/XAML and silently drops WM_CHAR, so the FG path is default — pass sendOptions:{method:'background'} only if your WT build accepts BG input.
Examples:
terminal({action:'run', windowTitle:'PowerShell', input:'npm test', until:{mode:'pattern', pattern:'Test Files'}}) → recommended for test runners; matches when vitest summary appears
terminal({action:'run', windowTitle:'pwsh', input:'ls'}) → quiet 1500ms wait, returns output (short interactive)
terminal({action:'run', windowTitle:'pwsh', command:'ls'}) → identical to the above; command is a deprecated alias of input (issue #245)
terminal({action:'read', windowTitle:'PowerShell', sinceMarker:'...'}) → incremental diff using the read action
terminal({action:'send', windowTitle:'PowerShell', input:'echo hello'}) → sends text + Enter using the send action
| Name | Required | Description | Default |
|---|---|---|---|
| input | No | 'send': Text to send (max 10,000 chars). 'run': Command to send (Enter is appended automatically). Either `input` or its deprecated alias `command` is required. | |
| lines | No | Tail N lines (default 50). | |
| until | No | ||
| action | Yes | Action selector — one of: read, send, run. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| method | No | Input routing channel. 'auto' defaults to background (WM_CHAR) when the target is a known terminal class (Windows Terminal / cmd / PowerShell / conhost) so user-side focus changes mid-stream cannot divert keystrokes. DTM_BG_AUTO=1 enables BG globally; 'auto' falls back to foreground for non-terminal targets. 'background' forces WM_CHAR injection (no focus change). 'foreground' forces the current behavior (SetForegroundWindow + clipboard paste). 'foreground_flash' (ADR-013 Option E) is an explicit opt-in 妥協 BG path for Windows Terminal: temporarily steals foreground (~50-80ms), pastes via clipboard, sends Ctrl+V + Enter (when pressEnter=true), restores foreground + clipboard. Single-line + < 5KiB only. `typingLeakRisk: true` in hints. Default 'auto'. | auto |
| paneId | No | 'read': Pane handle from key_locker launch_console — either the decimal hwnd of a classic console, or the 'wt:…' form for a Windows Terminal tab. Targets THIS pane even after its title changes; takes precedence over windowTitle. NOTE: read still resolves the pane's text by title under the hood, so it declines if the pane's current title is no longer unique among windows (a 'wt:…' pane additionally reads only while its tab is the ACTIVE tab of its Windows Terminal window). 'send': Pane handle from key_locker launch_console — either the decimal hwnd of a classic console (bound directly by hwnd, surviving a title change, e.g. after an ssh login the title becomes user@host), or the 'wt:…' form for a Windows Terminal tab (delivered to its Windows Terminal window while that tab is ACTIVE). Takes precedence over windowTitle. 'run': Pane handle from key_locker launch_console — a decimal console hwnd, or the `wt:<pid>:<startMs>` form for a Windows Terminal tab. Targets THIS pane for the whole run+wait+read: the pane title is re-resolved before every read, so a mid-run retitle (e.g. a classic console renaming to `user@host` after an ssh login) is still tracked. Takes precedence over windowTitle. This is the paneId FIELD of launch_console's result — NOT its windowTitle. A Windows Terminal pane is tracked while its tab is the ACTIVE tab (switching away pauses the reads until you switch back). | |
| source | No | 'auto' = UIA TextPattern then OCR fallback; 'uia' = TextPattern only (fail on miss); 'ocr' = OCR only. | auto |
| command | No | [Deprecated alias of `input`] Accepted for callers that mis-remember the parameter name; new code should use `input`. If both are set, `input` wins. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| pasteKey | No | Paste key combo. 'auto' picks ctrl+shift+v for WSL/bash/mintty/wezterm/alacritty, ctrl+v elsewhere. Only used when preferClipboard=true. | auto |
| settleMs | No | Milliseconds to wait after sending before checking foreground window (default 300). | |
| chunkSize | No | Split long input into chunks of this many characters in background mode to prevent terminal input queue saturation. Default 100. Only applies when method results in background. | |
| stripAnsi | No | Strip ANSI escape sequences (default true). | |
| timeoutMs | No | Hard timeout in ms (default 30s) | |
| focusFirst | No | Focus the terminal before sending (default true). | |
| forceFocus | No | When true, bypass Windows foreground-stealing protection via AttachThreadInput before focusing the terminal window. Default: follows env DESKTOP_TOUCH_FORCE_FOCUS (default false). | |
| pressEnter | No | Press Enter after typing (default true). | |
| trackFocus | No | When true (default), detect if focus was stolen after sending. Reports focusLost in the response. | |
| ocrLanguage | No | BCP-47 language tag for OCR fallback. Auto-detects from system locale when omitted. | |
| readOptions | No | Extra options forwarded to terminal read (lines, source, ocrLanguage, etc.) | |
| sendOptions | No | Extra options forwarded to terminal send (method, chunkSize, etc.) | |
| sinceMarker | No | Marker returned from a previous call. If found in current text, only the diff is returned. | |
| windowTitle | No | 'read' / 'run': Partial title of the terminal window (e.g. 'PowerShell', 'pwsh', 'WindowsTerminal'). Provide windowTitle OR paneId (paneId takes precedence). 'send': Partial title of the terminal window. Provide windowTitle OR paneId (paneId takes precedence). | |
| restoreFocus | No | Restore the previously-focused window after sending (default true). | |
| preferClipboard | No | Paste the input from the clipboard instead of typing it — safe for long text and not run through IME conversion (does not help while an IME composition is already in progress: the paste keystroke is consumed by the IME and nothing arrives, though the call still answers ok; hints.clipboard.imeOpen says when the IME was on at the paste). Your clipboard is replaced for the duration of the call and put back afterwards, but the restore is skipped when another process wrote to the clipboard first, when its contents are too large for an addon-less build to carry back, or when saving it failed; hints.clipboard reports which backend served the paste and whether the restore ran. Default true. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so: it enumerates completion reasons (quiet | pattern_matched | exited | timeout | window_closed | window_not_found | send_failed), documents outputIntegrity:'baseline_lost' and the forced empty output plus readError.code, the FileLockCollision warning, clipboard replacement by preferClipboard, the Windows Terminal exclusion from the BG path, and the foreground-refusal/other-desktop failure modes. Side effects (clipboard replaced, foreground stolen) and auth/routing constraints are surfaced up front.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded and cleanly sectioned with Purpose/Details/Prefer/Caveats/Examples, and the examples are useful. However, it is extremely dense and repeats material already carried by the 96%-coverage schema (paneId behavior is restated three times in the schema and again here, as is the clipboard-replacement caveat), so several sentences do not earn their place relative to structured fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, yet the description documents the response surface directly — completion={reason, elapsedMs}, outputIntegrity, warnings, hints (verifyDelivery.unverifiable, hints.clipboard, hints.terminalMarker.previousMatched) — along with recovery actions for each. For a 25-parameter, three-mode, multi-failure-mode tool, nothing operationally necessary is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 96%, so the baseline is 3, but the description adds genuinely non-redundant semantics: it explains the paneId-vs-windowTitle precedence ('pass the paneId field, NOT windowTitle'), the deprecated `command` alias for `input`, and when to choose pattern vs exit mode. It stops short of a full walkthrough of secondary parameters (settleMs, chunkSize, focusFirst), which the schema already covers.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a concrete verb+resource+scope: 'Interact with a terminal window: read output, send input, or run+wait+read in one call,' then splits the three action modes explicitly and notes that read/send absorb formerly-standalone tools. An agent can distinguish this from siblings like screenshot, keyboard, and window_dock without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit routing: prefer action='run' for execution+result, switch to until:{mode:'pattern'} for long-running builds/test runners because quiet mode can complete prematurely, and use read/send only for fine-grained interleaving. It also names the alternative to avoid ('Do not screenshot the terminal — action='read' is cheaper and structured'), which is exactly the when-not guidance the dimension asks for.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
wait_untilA
Purpose: Server-side poll for an observable condition — eliminates screenshot-polling loops when waiting for state changes. Details: condition selects what to watch: window_appears/window_disappears (target.windowTitle required), focus_changes (optional target.fromHwnd), element_appears/value_changes (target.windowTitle + target.elementName required, UIA; min 500ms interval), ready_state (target.windowTitle; visible + not minimized), terminal_output_contains (target.windowTitle + target.pattern required [+target.regex:true], needs terminal tools loaded), element_matches (target.by + target.pattern required, needs browser tools loaded), url_matches (target.pattern required [+target.regex:true]; matches the active tab's location.href via CDP — use for SPA route changes, redirects, OAuth flows). Returns {ok:true, elapsedMs, observed} on success, or WaitTimeout error with suggest hints. timeoutMs default 5000 (max 60000). Prefer: Use instead of run_macro({sleep:N}) + screenshot loops. Use terminal_output_contains to detect CLI command completion. Use element_matches for browser DOM readiness after navigation. Use url_matches when the URL is the most reliable signal (SPA routing / redirect cascades). Caveats: terminal_output_contains/element_matches/url_matches need a browser CDP connection (open --remote-debugging-port=9222 first). element_appears/value_changes spawn a UIA process per poll (interval floor 500ms). On timeout: {ok:false, code:'WaitTimeout', error, suggest:[...]}; suggest[] may open with a line earned by the last poll — read it, do not index. Those two also return context.lastLook: did the element resolve, why not, and for value_changes baseline/latest — the field's VALUE, so a masked credential arrives as mask characters. Other codes: 'ToolError' (validation / missing hook — read the message), 'BrowserNotConnected' (re-attach via browser_open), and 'WindowExcluded' for a target this server may not act through, answered at once rather than polled. Branch on code. Examples: wait_until({condition:'window_appears', target:{windowTitle:'Save As'}, timeoutMs:10000}) wait_until({condition:'terminal_output_contains', target:{windowTitle:'Terminal', pattern:'$ '}, timeoutMs:30000}) wait_until({condition:'element_matches', target:{by:'text', pattern:'Submit', scope:'#checkout-form'}}) wait_until({condition:'url_matches', target:{pattern:'/dashboard'}, timeoutMs:15000}) wait_until({condition:'url_matches', target:{pattern:'^https://app\\.example\\.com/orders/[0-9]+$', regex:true}})
| Name | Required | Description | Default |
|---|---|---|---|
| target | No | Target descriptor — fields used depend on condition. Accepts an object literal or a JSON-stringified object. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| condition | Yes | Condition to wait for. See per-condition target requirements. | |
| timeoutMs | No | Maximum time to wait (default 5000ms) | |
| intervalMs | No | Poll interval (default 200ms — terminal_output_contains uses 500 internally) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so: it discloses the success shape ({ok, elapsedMs, observed}), the four error codes (WaitTimeout, ToolError, BrowserNotConnected, WindowExcluded) and how to branch on them, the CDP connection prerequisite, and that element_appears/value_changes spawn a UIA process per poll. It even warns that suggest[] may open with a poll-earned line and that the lastLook baseline can surface masked credentials.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose/Details/Prefer/Caveats/Examples headers front-load the key information and the length is defensible for a nine-branch condition poller. Some caveat clauses are packed densely (the suggest-indexing and masked-value asides) and could be trimmed without losing call correctness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema and no annotations exist, yet the description still documents return values, timeout semantics, all failure codes, prerequisites, and per-condition target requirements. An agent has everything needed to select a condition, supply the right target fields, and handle the result.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% but the nested target object's fields carry no per-field descriptions in the schema, so the description's per-condition requirement mapping (which conditions need windowTitle, elementName, pattern, regex, by) is genuinely additive. It stops short of explaining by's role/ariaLabel values, port, tabId, or elementSelector, so a few nested fields remain unglossed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('server-side poll for an observable condition') and immediately positions itself against the screenshot-polling pattern agents typically fall back on. The nine condition branches make the scope unambiguous, and an agent can tell it apart from run_macro/screenshot siblings without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says 'Use instead of run_macro({sleep:N}) + screenshot loops' and then routes by scenario (terminal_output_contains for CLI completion, element_matches for DOM readiness, url_matches for SPA routing/redirect cascades). Both when-to-use and the named alternatives are present, so nothing is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
window_dockA
Purpose: Decorate a window: pin (always-on-top), unpin, or dock (move + resize + optional pin). Details: action='pin' makes window always-on-top until unpin/duration_ms. action='unpin' removes always-on-top. action='dock' positions to corner with width/height (default 480×360 bottom-right) and pins it unless pin:false. Minimized windows are automatically restored before docking. The title matches a shown window first; a hidden one (on another virtual desktop, or a minimised packaged app's content) is not docked or pinned — bring it back with focus_window first. Prefer: Use action='dock' for terminal/CLI window auto-positioning at session start. Use action='pin' alone when you only need always-on-top without moving or resizing. Caveats: Pin survives minimize/restore; explicit action='unpin' needed to release. Dock fails on elevated processes. Dock overrides any existing Win+Arrow snap arrangement. Examples: window_dock({action:'dock', title:'PowerShell', corner:'bottom-right', width:480, height:360}) window_dock({action:'pin', title:'Settings', duration_ms:5000}) window_dock({action:'unpin', title:'Settings'})
| Name | Required | Description | Default |
|---|---|---|---|
| pin | No | If true, set always-on-top so the docked window stays visible on top of other windows. Use window_dock(action='unpin') to remove the topmost flag later. Default true. | |
| title | No | 'pin' / 'unpin': Partial window title (case-insensitive) 'dock': Partial window title to dock (case-insensitive). Matches the first shown window containing this text (a minimised one counts; a hidden one — on another virtual desktop, or a minimised packaged app's content — does not). Example: 'Claude Code', 'メモ帳'. | |
| width | No | Window width in pixels after docking. Default 480. | |
| action | Yes | Action selector — one of: pin, unpin, dock. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional. | |
| corner | No | Screen corner to snap the window to. Default 'bottom-right'. | bottom-right |
| height | No | Window height in pixels after docking. Default 360. | |
| margin | No | Pixel padding between the window and the screen edge. Default 8. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| monitorId | No | Monitor to dock on (from desktop_state({includeScreen:true})). Omit for primary monitor. | |
| duration_ms | No | Auto-unpin after this many ms (0–60000). Omit to pin indefinitely. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so richly: pin survives minimize/restore and requires explicit unpin, minimized windows are auto-restored before docking, hidden windows are skipped, dock fails on elevated processes, and dock overrides existing Win+Arrow snap. These are exactly the side effects an agent needs before calling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Labeled sections (Purpose / Details / Prefer / Caveats / Examples) front-load the core purpose and keep each sentence earning its place. The three examples demonstrate distinct actions compactly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 10-parameter, no-annotation, no-output-schema mutation tool the coverage is strong: actions, defaults, and edge-case failure modes are all addressed. The only gap is that the response-shape opt-in (include/envelope) is left entirely to the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds essential per-action semantics the flat schema omits — the schema itself defers 'per-action required fields' to the description, and the Details section maps action to behavior (dock defaults 480×360 bottom-right, pin:false to skip topmost). It stops short of covering the include/response-shape opt-in.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening states a specific verb and resource ('Decorate a window: pin, unpin, or dock') and immediately disambiguates each action. It distinguishes itself from siblings by naming focus_window as the tool to use first for hidden windows, so an agent can route correctly without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer' block gives explicit when-to-use guidance: dock for terminal/CLI auto-positioning at session start, pin alone when only always-on-top is needed. It also names the alternative (focus_window) and the condition ('hidden one ... is not docked') that selects it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
workspace_launchA
Purpose: Launch an application and wait for its window to appear, returning the window's title, HWND, and PID. Details: Starts the executable (well-known names and App Paths entries resolve to a full path), lists the top-level windows before the launch, then checks every 200 ms, for up to waitMs, for a window whose HWND was not there before or whose title changed. Returns {launched, args, windowTitle, hwnd, pid, region, foundWindow}; the window fields are null when no window appeared in time, and a hint says why. foundWindow is the same as windowTitle, kept for older callers. Works for localized window titles (e.g. '電卓' for calc.exe) because detection is HWND-based. waitMs default 2000, max 30000; waitMs:0 returns right after the launch with the window fields null. Prefer: Use instead of run_macro({exec, sleep, desktop_discover}) combos. Follow with desktop_discover({target:{hwnd}}) using the returned hwnd, which names this window even when another has a similar title (or focus_window(windowTitle)). Caveats: Single-instance apps that reuse an existing window will not register as a new HWND — call desktop_discover first to check if the window is already open. Minimized windows and windows smaller than 50×50 are not reported. When no new window appears within waitMs, an already-open window whose title changed during the wait is reported instead, which may not be the launched app; such an app (one that only retitles its existing window) answers after the whole waitMs. pid is the process that owns the window, which can differ from the one launched (an app that hands off to another process). Examples: workspace_launch({command:'notepad.exe'}) → {windowTitle:'', hwnd:'...', pid:...} workspace_launch({command:'calc.exe', waitMs:15000})
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | Command-line arguments (max 20). Shell metacharacters (; & | ` $() ${}) are not allowed. | |
| waitMs | No | Milliseconds to wait for the window to appear (default 2000) | |
| command | Yes | Executable name or full path (e.g. 'notepad.exe', 'calc.exe'). Shell interpreters (cmd.exe, powershell.exe, etc.) are blocked. | |
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and delivers: polling cadence, null-window semantics, single-instance no-new-HWND behavior, minimized and <50x50 exclusion, the title-change fallback, and the fact that pid may differ from the launched process. This is exactly the behavioral context an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with Purpose then Details, and organized under Prefer/Caveats/Examples headers so an agent can scan it. It is on the long side and the caveat list is dense, but nearly every sentence encodes a distinct behavior rather than padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although there is no output schema, the description fully specifies the return object ({launched, args, windowTitle, hwnd, pid, region, foundWindow}) and notes the foundWindow/windowTitle alias and the 'hint' field. Nothing needed to call or interpret the result is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning beyond the schema: waitMs:0 returns immediately with null window fields, command resolves well-known names and App Paths entries to a full path, and localized titles are handled via HWND-based detection.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening states a specific verb and resource—launch an application and wait for its window—and enumerates the return fields (title, HWND, PID). It clearly distinguishes itself from siblings like run_macro and desktop_discover by naming what it replaces.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Prefer' section explicitly tells the agent to use this instead of run_macro({exec, sleep, desktop_discover}) combos, and prescribes follow-up calls (desktop_discover with the returned hwnd, or focus_window). Both the condition and the alternatives are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
workspace_snapshotA
Purpose: Orient fully in one call — returns display layouts, all window thumbnails (WebP), and per-window actionable element lists with clickAt coords. Details: uiSummary.actionable[] per window includes: action ('click'|'type'|'expand'|'select'), clickAt {x,y} (pass directly to mouse_click), value (current text for editable fields). Runs parallel internally; latency ≈ max(single screenshot), not N×screenshots. Also resets the diffMode buffer so subsequent screenshot(diffMode=true) returns only changes (P-frame). Prefer: Use at session start or after major workspace changes. Use screenshot(detail='meta') for cheap re-orientation within a session. Use screenshot(detail='text', windowTitle=X) for a single-window update. Caveats: Thumbnails are scaled, not 1:1 — use screenshot(dotByDot=true, windowTitle=X) for pixel-accurate coords on a specific window after snapshot. Also: this call resets the screenshot diff baseline (I-frame) and identity tracker as a side effect, so subsequent screenshot(diffMode=true) starts fresh from this snapshot. The reset is not currently exposed in causal/working memory — record an explicit 'workspace_snapshot' step if you need to track the reset point in your causal trail (ADR-010 §11 OQ carry-over for full visibility).
| Name | Required | Description | Default |
|---|---|---|---|
| include | No | Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients). | |
| includeUiSummary | No | Whether to include UI element summaries for each window | |
| thumbnailMaxDimension | No | Max size of per-window thumbnail images (default 400px) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden and does so well: it discloses internal parallelism, latency characteristics (≈ max single screenshot), scaled-not-1:1 thumbnails, and a significant side effect (resetting the screenshot diff baseline/identity tracker). It even flags that the reset isn't reflected in causal memory, which is rare and useful context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well front-loaded with labeled sections (Purpose, Details, Prefer, Caveats) that let an agent scan quickly. However, it is verbose and the diff-baseline reset is stated twice (once as 'resets the diffMode buffer', again as 'resets the screenshot diff baseline'), and jargon like 'ADR-010 §11 OQ carry-over' adds length without actionable value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly explains the return shape (layouts, thumbnails, actionable element lists with coords) and covers the side effects and caveats an agent needs before calling. Nothing material is missing for a zero-required-parameter orientation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the three input parameters (include, includeUiSummary, thumbnailMaxDimension) are already fully documented in the schema. The description adds no input-parameter semantics beyond that; its detailed field talk (clickAt, actionable[]) describes response content, not inputs. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line states a specific verb ('Orient fully') and enumerates exactly what is returned: display layouts, WebP thumbnails, and per-window actionable element lists with clickAt coords. It is clearly distinguishable from screenshot siblings by naming the alternatives (screenshot(detail='meta'), screenshot(detail='text')) and contrasting their scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use it ('session start or after major workspace changes') and routes each alternative case to the correct sibling tool ('screenshot(detail=meta)' for cheap re-orientation, 'screenshot(detail=text, windowTitle=X)' for a single window). This is textbook when/when-not/alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
30 tool updates
v2.3.1- Changed
browser_click9 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / by / descriptionPrevious value: -"Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with pattern. Resolves to a SINGLE actionable element and STOPS with candidates when ambiguous."New value: +"Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with `pattern`. The server resolves to a SINGLE actionable element — climbing to a clickable ancestor up to 3 levels — and STOPS with a candidate list (code:'BrowserAmbiguousTarget') if the match is ambiguous; it never guesses." - changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."New value: +"Narration level. \"rich\": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — \"@active\" and the dialog rescue both resolve one (\"ambiguous_title\"); or the target moved between the snapshot and the action (\"target_changed\"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see (\"fix_target_unknown\") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too (\"tree_truncated\"). The action itself is unaffected in every case; only the diff is. Default: \"minimal\"." - changed
Input schema / properties / pattern / descriptionPrevious value: -"Value matched against the chosen by axis (required when by is set)."New value: +"Value matched against the chosen `by` axis (required when `by` is set)." - added
Input schema / properties / pattern / minLengthAdded value: +1 - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)" - changed
Input schema / properties / role / descriptionPrevious value: -"Optional ARIA/implicit-role filter AND-combined with by (e.g. by:'text', pattern:'Save', role:'button')."New value: +"Optional ARIA/implicit-role filter AND-combined with `by` (e.g. by:'text', pattern:'Save', role:'button')." - changed
Input schema / properties / scope / descriptionPrevious value: -"Optional CSS selector to limit the by-axis search scope (disambiguation)."New value: +"Optional CSS selector to limit the `by`-axis search scope (disambiguation)."
- Changed
browser_eval5 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / includeContext / descriptionPrevious value: -"When true, append activeTab and readyState context to the response."New value: +"When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab." - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)" - added
Input schema / properties / selectors / itemsAdded value: +{ + "type": "string" +}
- Changed
browser_fill9 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / by / descriptionPrevious value: -"Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with pattern. Resolves to a SINGLE actionable element and STOPS with candidates when ambiguous."New value: +"Semantic axis to target by INSTEAD of a CSS selector: 'text' (visible text), 'regex', 'role' (ARIA/implicit role), 'ariaLabel'. Pair with `pattern`. The server resolves to a SINGLE actionable element — climbing to a clickable ancestor up to 3 levels — and STOPS with a candidate list (code:'BrowserAmbiguousTarget') if the match is ambiguous; it never guesses." - changed
Input schema / properties / includeContext / descriptionPrevious value: -"When true, append activeTab and readyState context to the response."New value: +"When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab." - changed
Input schema / properties / pattern / descriptionPrevious value: -"Value matched against the chosen by axis (required when by is set)."New value: +"Value matched against the chosen `by` axis (required when `by` is set)." - added
Input schema / properties / pattern / minLengthAdded value: +1 - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)" - changed
Input schema / properties / role / descriptionPrevious value: -"Optional ARIA/implicit-role filter AND-combined with by (e.g. by:'text', pattern:'Save', role:'button')."New value: +"Optional ARIA/implicit-role filter AND-combined with `by` (e.g. by:'text', pattern:'Save', role:'button')." - changed
Input schema / properties / scope / descriptionPrevious value: -"Optional CSS selector to limit the by-axis search scope (disambiguation)."New value: +"Optional CSS selector to limit the `by`-axis search scope (disambiguation)."
- Changed
browser_form4 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / includeContext / descriptionPrevious value: -"When true, append activeTab and readyState context to the response."New value: +"When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab." - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)"
- Changed
browser_locate5 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / includeContext / descriptionPrevious value: -"When true, append activeTab and readyState context to the response."New value: +"When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab." - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)" - changed
Input schema / properties / selector / descriptionPrevious value: -"CSS selector for the target element (e.g. '#submit', '.btn', 'button[type=submit]')."New value: +"CSS selector for the target element (e.g. '#submit', '.btn', 'button[type=submit]')"
- Changed
browser_navigate4 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."New value: +"Narration level. \"rich\": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — \"@active\" and the dialog rescue both resolve one (\"ambiguous_title\"); or the target moved between the snapshot and the action (\"target_changed\"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see (\"fix_target_unknown\") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too (\"tree_truncated\"). The action itself is unaffected in every case; only the diff is. Default: \"minimal\"." - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)"
- Changed
browser_open4 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - removed
Input schema / properties / launch / additionalPropertiesRemoved value: -false - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)"
- Changed
browser_overview4 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / includeContext / descriptionPrevious value: -"When true, append activeTab and readyState context to the response."New value: +"When true (default), append `activeTab` + `readyState` lines to the response. Set false to skip — saves ~150 tokens per call when chaining several browser_* calls in the same tab." - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)"
- Changed
browser_search4 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / offset / maximumAdded value: +9007199254740991 - changed
Input schema / properties / port / descriptionPrevious value: -"Chrome/Edge CDP remote debugging port."New value: +"Chrome/Edge CDP remote debugging port (default 9222; configurable via desktop-touch-config.json)"
- Changed
click_element3 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."New value: +"Narration level. \"rich\": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — \"@active\" and the dialog rescue both resolve one (\"ambiguous_title\"); or the target moved between the snapshot and the action (\"target_changed\"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see (\"fix_target_unknown\") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too (\"tree_truncated\"). The action itself is unaffected in every case; only the diff is. Default: \"minimal\"."
- Changed
clipboard2 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false
- Changed
desktop_state2 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false
- Changed
excel8 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -true - added
Input schema / properties / actionAdded value: +{ + "description": "Action selector — one of: run_vba, check_access_vbom. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional.", + "enum": [ + "run_vba", + "check_access_vbom" + ], + "type": "string" +} - added
Input schema / properties / codeAdded value: +{ + "description": "VBA source. MUST declare at least one Sub matching `macroName` (default `DesktopTouchAdHoc`). Example: \n\"Sub DesktopTouchAdHoc()\\r\\n Range(\\\"A1\\\").Value = \\\"Hello\\\"\\r\\nEnd Sub\"", + "minLength": 1, + "type": "string" +} - added
Input schema / properties / includeAdded value: +{ + "description": "Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients).", + "items": { + "type": "string" + }, + "type": "array" +} - added
Input schema / properties / macroNameAdded value: +{ + "default": "DesktopTouchAdHoc", + "description": "Sub name to invoke. MUST appear in `code` as `Sub <name>(...)`. Default `DesktopTouchAdHoc`.", + "minLength": 1, + "type": "string" +} - added
Input schema / properties / visibleAdded value: +{ + "default": false, + "description": "If true, show the Excel window during execution. Default false (headless). Setting visible:true is useful for demo recording but may surface MsgBox/InputBox calls in the macro that block the COM thread.", + "type": "boolean" +} - added
Input schema / requiredAdded value: +[ + "action" +]
- Changed
focus_window2 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false
- Changed
key_locker12 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -true - added
Input schema / properties / actionAdded value: +{ + "description": "Action selector — one of: list, save, forget, set_policy, status, launch_console, fill_once. Per-action required fields are enforced at call time (see the tool description); this flat schema lists every action's fields as optional.", + "enum": [ + "list", + "save", + "forget", + "set_policy", + "status", + "launch_console", + "fill_once" + ], + "type": "string" +} - added
Input schema / properties / confirmEveryInjectionAdded value: +{ + "description": "true (default) = confirm every autofill for this binding; false = opt out (autofill without asking).", + "type": "boolean" +} - added
Input schema / properties / freshAdded value: +{ + "description": "false (default) = reuse the most-recent still-open anchored pane of the requested host; true = open a NEW one (bounded).", + "type": "boolean" +} - added
Input schema / properties / hostAdded value: +{ + "description": "'windows-terminal' (default) = open a new tab in the user's current Windows Terminal window. 'classic' = open a dedicated classic console window instead (the fallback when Windows Terminal is not installed — a KeyLockerWtUnavailable error tells you to retry with this). A Windows Terminal pane autofills/reads only while its tab is the ACTIVE tab; a classic console has its own window.", + "enum": [ + "windows-terminal", + "classic" + ], + "type": "string" +} - added
Input schema / properties / includeAdded value: +{ + "description": "Optional response-shape opt-in. `['envelope']` returns the self-documenting envelope (`_version` / `data` / `as_of` / `confidence`). `['raw']` forces raw shape (overrides DESKTOP_TOUCH_ENVELOPE=1 server default). Default behaviour is raw shape (compat with existing clients).", + "items": { + "type": "string" + }, + "type": "array" +} - added
Input schema / properties / paneIdAdded value: +{ + "description": "The paneId returned by launch_console (only those panes can be filled).", + "minLength": 1, + "type": "string" +} - added
Input schema / properties / reasonAdded value: +{ + "description": "One short sentence the dialog shows the user as YOUR reason (e.g. 'log in to 192.168.1.20 via the jump host'). Shown as the assistant's words; never put a secret here.", + "maxLength": 200, + "minLength": 1, + "type": "string" +} - added
Input schema / properties / submitAdded value: +{ + "description": "true (default) = press Enter after typing the secret; false = type only.", + "type": "boolean" +} - added
Input schema / properties / uriAdded value: +{ + "description": "A binding URI identifying the credential target — e.g. `ssh://user@host:22`, `sudo://host/root`, `https-cred://github.com:443`, `sshkey:SHA256:…` (no `//`). Parsed per the L1 grammar; malformed input is rejected.", + "minLength": 1, + "type": "string" +} - added
Input schema / requiredAdded value: +[ + "action" +]
- Changed
keyboard16 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / fixId / descriptionPrevious value: -"Approve a pending suggestedFix (one-shot, 15s TTL). Pass the fixId returned by a previous failed keyboard(action='type') to re-attempt with guard-validated args."New value: +"'type': Approve a pending suggestedFix (one-shot, 15s TTL). Pass the fixId returned by a previous failed keyboard(action='type') to re-attempt with guard-validated args.\n'sequence': Approve a pending suggestedFix (one-shot, 15s TTL). Only meaningful for GUARD-pre-loop rejections (e.g. unsafe.keyboardTarget). Mid-loop MenuFocusLostMidSequence does NOT issue fixIds — recover by re-calling with context.remaining." - changed
Input schema / properties / forceFocus / descriptionPrevious value: -"Bypass Windows foreground-stealing protection before focusing."New value: +"When true, bypass Windows foreground-stealing protection via AttachThreadInput before focusing the target window. Default: follows env DESKTOP_TOUCH_FORCE_FOCUS (default false)." - changed
Input schema / properties / forceImeOff / descriptionPrevious value: -"Issue #245 系統②: when true, query the target window's IME open-status via Imm32 before typing; if ON, switch OFF for the duration of this call and restore the prior state in `finally`. Prevents silent romaji conversion when the user's Japanese IME is active but the LLM is typing ASCII commands. Requires `windowTitle` or `hwnd` (otherwise no target to query). Default false — existing use_clipboard auto-promotion still handles non-ASCII symbols transparently. No-op when the addon predates the IMM bridge (call proceeds with whatever IME state is in effect)."New value: +"'type': Issue #245 系統②: when true, query the target window's IME open-status via Imm32 before typing; if ON, switch OFF for the duration of this call and restore the prior state in `finally`. Prevents silent romaji conversion when the user's Japanese IME is active but the LLM is typing ASCII commands. Requires `windowTitle` or `hwnd` (otherwise no target to query). Default false — existing use_clipboard auto-promotion still handles non-ASCII symbols transparently. No-op when the addon predates the IMM bridge (call proceeds with whatever IME state is in effect).\n'sequence': Issue #245 系統②: query the target's IME open-status before the first step; if ON, switch OFF for the whole sequence and restore in finally. Prevents Alt-mnemonic hijack when 日本語 IME is active (the OS routes Alt+letter to IME composition instead of the menu). Requires windowTitle or hwnd. Default false." - changed
Input schema / properties / hwnd / descriptionPrevious value: -"Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues."New value: +"Direct window handle ID (takes precedence over windowTitle). Either this or windowTitle is required. Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues. A window that has no title can be addressed this way, but only while it is already the foreground window — keyboard focus and guarding cannot target a titleless window yet." - changed
Input schema / properties / lensId / descriptionPrevious value: -"Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated before typing, and a perception envelope is attached to post.perception on success."New value: +"'type': Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated before typing, and a perception envelope is attached to post.perception on success.\n'press': Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated before the key press.\n'sequence': Optional perception lens ID. Guards (safe.keyboardTarget) are evaluated once before the first step." - added
Input schema / properties / method / anyOfAdded value: +[ + { + "enum": [ + "auto", + "background", + "foreground", + "foreground_flash" + ], + "type": "string" + }, + { + "const": "foreground", + "type": "string" + } +] - removed
Input schema / properties / method / defaultRemoved value: -"auto" - changed
Input schema / properties / method / descriptionPrevious value: -"Input method. background = WM_CHAR PostMessage (no focus change); foreground = SendInput (current default); auto = pick automatically."New value: +"'type' / 'press': Input routing channel. 'auto' uses background (PostMessage) when the target window is a known terminal class (Windows Terminal / cmd / PowerShell) OR DTM_BG_AUTO=1 is set; else foreground. Terminal auto-detect is HWND-targeted so user-side focus changes mid-stream cannot divert keystrokes (except a combo with ctrl/shift/alt, or type with replaceAll, which 'auto' sends through the foreground). 'background' forces PostMessage-only (no focus change, fails on Chromium/IME). A key combo with ctrl, shift or alt, and type's replaceAll (which selects with Ctrl+A), cannot be posted — the app would see the plain key and type it — so 'background' refuses them with BackgroundModifierComboUnsupported and nothing is sent; 'auto' sends them through the foreground. 'foreground' forces the current behavior (SetForegroundWindow + keystrokes). 'foreground_flash' (ADR-013 Option E) is an explicit opt-in 妥協 BG path for Windows Terminal: temporarily steals foreground (~50-80ms), pastes via clipboard, sends Ctrl+V, restores foreground + clipboard. Single-line + < 5KiB only. Carries `typingLeakRisk: true` in hints because user keystrokes during the flash window can leak to WT. Default 'auto'.\n'sequence': Sequence is foreground-only by design — Alt-menu mnemonics need real SendInput. Omit, or pass 'foreground'. method:'background' / 'foreground_flash' are rejected at schema parse time (typed codes BackgroundNotApplicableToSequence / ForegroundFlashNotApplicableToSequence document the rationale for LLMs)." - removed
Input schema / properties / method / enumRemoved value: -[ - "auto", - "background", - "foreground" -] - removed
Input schema / properties / method / typeRemoved value: -"string" - changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."New value: +"Narration level. \"rich\": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — \"@active\" and the dialog rescue both resolve one (\"ambiguous_title\"); or the target moved between the snapshot and the action (\"target_changed\"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see (\"fix_target_unknown\") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too (\"tree_truncated\"). The action itself is unaffected in every case; only the diff is. Default: \"minimal\"." - changed
Input schema / properties / settleMs / descriptionPrevious value: -"Milliseconds to wait before checking post-action state."New value: +"Milliseconds to wait after the action before checking foreground window (default 300)." - changed
Input schema / properties / trackFocus / descriptionPrevious value: -"Detect if focus was stolen after the action."New value: +"When true (default), detect if focus was stolen from the target window after the action. Reports focusLost in the response. Set false to skip." - changed
Input schema / properties / windowTitle / descriptionPrevious value: -"Partial title of the window that should receive keyboard input."New value: +"Partial title of the window that should receive the keystrokes. Required unless you pass hwnd: a call with neither stops with DestinationRequired before any key is sent, and an empty or whitespace-only value counts as neither. The server focuses this window before typing and uses it as the expected target for focusLost detection. Use '@active' to target the current foreground window on purpose."
- Changed
mouse_click19 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / elementId / descriptionPrevious value: -"AutomationId of the UI element."New value: +"Hint: automationId of the UI element (from actionable[].id). Requires windowTitle. Used with elementName for more precise UIA re-query." - changed
Input schema / properties / elementName / descriptionPrevious value: -"Name or label of the UI element."New value: +"Hint: name/label of the UI element (from actionable[].name in screenshot(detail='text')). Requires windowTitle. Triggers UIA re-query to get fresh coordinates when the window resized or moved far." - changed
Input schema / properties / forceFocus / descriptionPrevious value: -"Bypass Windows foreground-stealing protection before focusing."New value: +"When true, bypass Windows foreground-stealing protection via AttachThreadInput before focusing the target window. Required when a pinned window (e.g. Claude CLI) keeps stealing focus. Default: follows env DESKTOP_TOUCH_FORCE_FOCUS (default false). Set DESKTOP_TOUCH_FORCE_FOCUS=1 to make true the global default." - changed
Input schema / properties / homing / descriptionPrevious value: -"Enable homing correction if the target window moved."New value: +"Enable homing correction (default true). When enabled, the MCP server corrects stale coordinates if the target window moved since the last screenshot. Set false to disable all correction (like traction control OFF)." - changed
Input schema / properties / hwnd / descriptionPrevious value: -"Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues."New value: +"Direct window handle ID (takes precedence over windowTitle). Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues." - changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."New value: +"Narration level. \"rich\": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — \"@active\" and the dialog rescue both resolve one (\"ambiguous_title\"); or the target moved between the snapshot and the action (\"target_changed\"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see (\"fix_target_unknown\") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too (\"tree_truncated\"). The action itself is unaffected in every case; only the diff is. Default: \"minimal\"." - removed
Input schema / properties / origin / additionalPropertiesRemoved value: -false - added
Input schema / properties / scale / exclusiveMinimumAdded value: +0 - changed
Input schema / properties / settleMs / descriptionPrevious value: -"Milliseconds to wait before checking post-action state."New value: +"Milliseconds to wait after the action before checking foreground window (default 300). Only used when trackFocus=true." - changed
Input schema / properties / speed / descriptionPrevious value: -"Cursor movement speed in px/sec. 0 = instant."New value: +"Cursor movement speed in px/sec. 0 = instant (teleport, no animation). Omit to use the configured default (DESKTOP_TOUCH_MOUSE_SPEED env var, default 3000)." - added
Input schema / properties / speed / maximumAdded value: +9007199254740991 - changed
Input schema / properties / trackFocus / descriptionPrevious value: -"Detect if focus was stolen after the action."New value: +"When true (default), detect if focus was stolen from the target window after the action. Reports focusLost:{afterMs,expected,stolenBy,stolenByProcessName} in the response. Set false to skip the settle wait and focus check." - added
Input schema / properties / verifyDelivery / defaultAdded value: +true - changed
Input schema / properties / verifyDelivery / descriptionPrevious value: -"Parameter 'verifyDeliveryParam' from the Windows server schema."New value: +"When true (default), capture pre/post snapshots of element-under-cursor + focusedElement + foregroundWindow + scrollPos to populate hints.verifyDelivery with status='delivered' | 'focus_only' | 'unverifiable' (issue #178). Set false to skip the extra UIA work when the caller will read post state itself." - added
Input schema / properties / verifyDelivery / typeAdded value: +"boolean" - changed
Input schema / properties / windowTitle / descriptionPrevious value: -"Partial title of the target window."New value: +"Hint: partial title of the window being clicked. Enables window-delta correction and auto-focus if the window went behind another. Use '@active' for the current foreground window. Example: \"メモ帳\", \"Google Chrome\"" - changed
Input schema / requiredPrevious value: -[ - "x", - "y", - "verifyDelivery" -]New value: +[ + "x", + "y" +]
- Changed
mouse_drag12 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / homing / descriptionPrevious value: -"Enable homing correction if the target window moved."New value: +"Enable homing correction (default true). When enabled, the MCP server corrects stale coordinates if the target window moved since the last screenshot. Set false to disable all correction (like traction control OFF)." - changed
Input schema / properties / hwnd / descriptionPrevious value: -"Direct window handle ID (takes precedence over windowTitle). Obtain from get_windows response (hwnd field). String type to avoid 64-bit precision issues."New value: +"Direct window handle ID (takes precedence over windowTitle). Obtain from desktop_discover response (windows[].hwnd). String type to avoid 64-bit precision issues." - changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."New value: +"Narration level. \"rich\": include UIA diff in post.rich (appeared/disappeared/valueDeltas, and nameDeltas when an element's name changed, such as a calculator display) — usually removes the need for a verification screenshot. It is withheld, with post.rich.diffDegraded saying why, when the diff cannot be shown to describe the window that was acted on. On click_element and keyboard (and set_element_value where the server registers it), which resolve the target window before acting: another open window's title contains the text this call resolved to, whether you named the hwnd or the server did — \"@active\" and the dialog rescue both resolve one (\"ambiguous_title\"); or the target moved between the snapshot and the action (\"target_changed\"). On those and on mouse_click, retrying with a fixId also withholds it, because the stored fix names a window this cannot see (\"fix_target_unknown\") — browser tools keep their diff on a fixId retry, because theirs is a tab diff and does not depend on a window title. mouse_click and mouse_drag accept an hwnd and still take their snapshots by windowTitle, so nothing else is withheld for them: verify those with a screenshot. A window too large to read whole is withheld too (\"tree_truncated\"). The action itself is unaffected in every case; only the diff is. Default: \"minimal\"." - changed
Input schema / properties / speed / descriptionPrevious value: -"Cursor movement speed in px/sec. 0 = instant."New value: +"Cursor movement speed in px/sec. 0 = instant (teleport, no animation). Omit to use the configured default (DESKTOP_TOUCH_MOUSE_SPEED env var, default 3000)." - added
Input schema / properties / speed / maximumAdded value: +9007199254740991 - added
Input schema / properties / verifyDelivery / defaultAdded value: +true - changed
Input schema / properties / verifyDelivery / descriptionPrevious value: -"Parameter 'verifyDeliveryParam' from the Windows server schema."New value: +"When true (default), capture pre/post snapshots of element-under-cursor + focusedElement + foregroundWindow + scrollPos to populate hints.verifyDelivery with status='delivered' | 'focus_only' | 'unverifiable' (issue #178). Set false to skip the extra UIA work when the caller will read post state itself." - added
Input schema / properties / verifyDelivery / typeAdded value: +"boolean" - changed
Input schema / properties / windowTitle / descriptionPrevious value: -"Partial title of the target window."New value: +"Hint: partial title of the window being clicked. Enables window-delta correction and auto-focus if the window went behind another. Use '@active' for the current foreground window. Example: \"メモ帳\", \"Google Chrome\"" - changed
Input schema / requiredPrevious value: -[ - "startX", - "startY", - "endX", - "endY", - "verifyDelivery" -]New value: +[ + "startX", + "startY", + "endX", + "endY" +]
- Changed
notification_show2 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false
- Changed
run_macro8 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - removed
Input schema / properties / steps / items / additionalPropertiesRemoved value: -false - added
Input schema / properties / steps / items / properties / params / additionalPropertiesAdded value: +{} - added
Input schema / properties / steps / items / properties / params / defaultAdded value: +{} - added
Input schema / properties / steps / items / properties / params / propertyNamesAdded value: +{ + "type": "string" +} - added
Input schema / properties / steps / items / properties / tool / descriptionAdded value: +"Tool name to call. One of: desktop_state, screenshot, mouse_click, mouse_drag, click_element, focus_window, keyboard, clipboard, window_dock, scroll, terminal, browser_open, browser_eval, browser_search, browser_overview, browser_locate, browser_click, browser_navigate, browser_fill, browser_form, workspace_snapshot, workspace_launch, wait_until, notification_show, desktop_discover, desktop_act, or the special pseudo-command \"sleep\"." - added
Input schema / requiredAdded value: +[ + "steps" +]
- Changed
screenshot12 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / displayId / maximumAdded value: +9007199254740991 - changed
Input schema / properties / dotByDot / descriptionPrevious value: -"1:1 pixel mode — no scaling, WebP compression. Window captures include 'origin: (x,y)' so you can compute screen position: screen_x = origin_x + image_x. A frame from the window's composition surface (WGC: mode='background' with fullContent on a visible, non-minimised window when it can be, or normal mode's rescue) starts at the window's visible frame, and its origin is that frame's corner; when the visible frame cannot be read, no origin is given — the text says so; use desktop_discover for coordinates. When dotByDotMaxDimension is also set, scale factor is included: screen_x = origin_x + image_x / scale."New value: +"1:1 pixel mode — no scaling, WebP compression. Window captures include 'origin: (x,y)' so you can compute screen position: screen_x = origin_x + image_x. A frame from the window's composition surface (WGC: mode='background' with fullContent on a window visible on the current desktop — not minimised or cloaked — when it can be, or normal mode's rescue) starts at the window's visible frame, and its origin is that frame's corner; when the visible frame cannot be read, no origin is given — the text says so; use desktop_discover for coordinates. When dotByDotMaxDimension is also set, scale factor is included: screen_x = origin_x + image_x / scale." - added
Input schema / properties / dotByDotMaxDimension / exclusiveMinimumAdded value: +0 - added
Input schema / properties / dotByDotMaxDimension / maximumAdded value: +9007199254740991 - added
Input schema / properties / maxDimension / exclusiveMinimumAdded value: +0 - added
Input schema / properties / maxDimension / maximumAdded value: +9007199254740991 - changed
Input schema / properties / mode / descriptionPrevious value: -"Capture mode.\n 'normal' — default. Window-targeted captures (windowTitle / hwnd) use Win32 PrintWindow with automatic BitBlt fallback when PrintWindow returns no data or an all-black frame; the route used is reported in hints.captureSource. Fullscreen / displayId captures use BitBlt.\n 'background' — a capture of the window itself, not of the screen: its composition surface (WGC) when it can be for a visible, non-minimised window with fullContent, else Win32 PrintWindow. Requires windowTitle (or hwnd). Pair with fullContent for GPU-rendered apps."New value: +"Capture mode.\n 'normal' — default. Window-targeted captures (windowTitle / hwnd) use Win32 PrintWindow; when it returns no data or an all-black frame, the window's composition surface (WGC) is tried next if the window is visible on the current desktop (not minimised or cloaked), and BitBlt last. The route used is reported in hints.captureSource ('printwindow' | 'wgc' | 'bitblt-fallback'). Fullscreen / displayId captures use BitBlt.\n 'background' — a capture of the window itself, not of the screen: its composition surface (WGC) when it can be for a window visible on the current desktop (not minimised or cloaked) with fullContent, else Win32 PrintWindow. Requires windowTitle (or hwnd). Pair with fullContent for GPU-rendered apps." - removed
Input schema / properties / region / additionalPropertiesRemoved value: -false - added
Input schema / properties / region / properties / height / exclusiveMinimumAdded value: +0 - added
Input schema / properties / region / properties / width / exclusiveMinimumAdded value: +0
- Changed
screenshot_gc5 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / maxAgeMs / maximumAdded value: +9007199254740991 - added
Input schema / properties / maxCount / maximumAdded value: +9007199254740991 - added
Input schema / properties / maxTotalBytes / maximumAdded value: +9007199254740991
- Changed
screenshot_query7 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / offset / maximumAdded value: +9007199254740991 - added
Input schema / properties / since / maximumAdded value: +9007199254740991 - added
Input schema / properties / since / minimumAdded value: +-9007199254740991 - added
Input schema / properties / until / maximumAdded value: +9007199254740991 - added
Input schema / properties / until / minimumAdded value: +-9007199254740991
- Changed
scroll11 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - changed
Input schema / properties / direction / descriptionPrevious value: -"Scroll direction"New value: +"'raw': Scroll direction\n'smart': Scroll direction. into-view: scroll until target element is visible (default). Other values scroll unconditionally.\n'capture': Scroll direction: 'down' (vertical, uses Page Down key) or 'right' (horizontal, uses mouse scroll). Default 'down'." - added
Input schema / properties / maxWidth / exclusiveMinimumAdded value: +0 - added
Input schema / properties / maxWidth / maximumAdded value: +9007199254740991 - changed
Input schema / properties / port / descriptionPrevious value: -"CDP port for Chrome path (default 9222)"New value: +"'to_element': CDP port for Chrome path (default 9222)\n'smart': CDP port (default 9222)" - changed
Input schema / properties / scrollDelayMs / descriptionPrevious value: -"Milliseconds to wait after each scroll for rendering to settle (default 400). Increase for slow/animated pages."New value: +"'capture': Milliseconds to wait after each scroll for rendering to settle (default 400). Increase for slow/animated pages.\n'read': Milliseconds to wait after each scroll for rendering to settle (default 400)." - changed
Input schema / properties / tabId / descriptionPrevious value: -"Tab ID (Chrome path only). Omit for first page tab."New value: +"'to_element': Tab ID (Chrome path only). Omit for first page tab.\n'smart': CDP tab ID (Chrome path only). Omit for first page tab." - added
Input schema / properties / virtualIndex / maximumAdded value: +9007199254740991 - added
Input schema / properties / virtualTotal / maximumAdded value: +9007199254740991 - changed
Input schema / properties / windowTitle / descriptionPrevious value: -"Partial window title. When provided, the server focuses this window first."New value: +"'raw': Partial window title. When provided, the server focuses this window first.\n'to_element': Partial window title (required for native path when name is used)\n'smart': Partial window title. Required for UIA and image paths. For CDP path, optional.\n'capture': Partial title of the window to capture (case-insensitive match)\n'read': Partial window title to focus and OCR (case-insensitive match)."
- Changed
server_status2 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false
- Changed
terminal27 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / chunkSizeAdded value: +{ + "default": 100, + "description": "Split long input into chunks of this many characters in background mode to prevent terminal input queue saturation. Default 100. Only applies when method results in background.", + "maximum": 10000, + "minimum": 1, + "type": "integer" +} - added
Input schema / properties / focusFirstAdded value: +{ + "default": true, + "description": "Focus the terminal before sending (default true).", + "type": "boolean" +} - added
Input schema / properties / forceFocusAdded value: +{ + "description": "When true, bypass Windows foreground-stealing protection via AttachThreadInput before focusing the terminal window. Default: follows env DESKTOP_TOUCH_FORCE_FOCUS (default false).", + "type": "boolean" +} - changed
Input schema / properties / input / descriptionPrevious value: -"Command to send (Enter is appended automatically). Either `input` or its deprecated alias `command` is required."New value: +"'send': Text to send (max 10,000 chars).\n'run': Command to send (Enter is appended automatically). Either `input` or its deprecated alias `command` is required." - added
Input schema / properties / linesAdded value: +{ + "default": 50, + "description": "Tail N lines (default 50).", + "maximum": 2000, + "minimum": 1, + "type": "integer" +} - added
Input schema / properties / methodAdded value: +{ + "default": "auto", + "description": "Input routing channel. 'auto' defaults to background (WM_CHAR) when the target is a known terminal class (Windows Terminal / cmd / PowerShell / conhost) so user-side focus changes mid-stream cannot divert keystrokes. DTM_BG_AUTO=1 enables BG globally; 'auto' falls back to foreground for non-terminal targets. 'background' forces WM_CHAR injection (no focus change). 'foreground' forces the current behavior (SetForegroundWindow + clipboard paste). 'foreground_flash' (ADR-013 Option E) is an explicit opt-in 妥協 BG path for Windows Terminal: temporarily steals foreground (~50-80ms), pastes via clipboard, sends Ctrl+V + Enter (when pressEnter=true), restores foreground + clipboard. Single-line + < 5KiB only. `typingLeakRisk: true` in hints. Default 'auto'.", + "enum": [ + "auto", + "background", + "foreground", + "foreground_flash" + ], + "type": "string" +} - added
Input schema / properties / ocrLanguageAdded value: +{ + "description": "BCP-47 language tag for OCR fallback. Auto-detects from system locale when omitted.", + "maxLength": 20, + "type": "string" +} - added
Input schema / properties / paneId / descriptionAdded value: +"'read': Pane handle from key_locker launch_console — either the decimal hwnd of a classic console, or the 'wt:…' form for a Windows Terminal tab. Targets THIS pane even after its title changes; takes precedence over windowTitle. NOTE: read still resolves the pane's text by title under the hood, so it declines if the pane's current title is no longer unique among windows (a 'wt:…' pane additionally reads only while its tab is the ACTIVE tab of its Windows Terminal window).\n'send': Pane handle from key_locker launch_console — either the decimal hwnd of a classic console (bound directly by hwnd, surviving a title change, e.g. after an ssh login the title becomes user@host), or the 'wt:…' form for a Windows Terminal tab (delivered to its Windows Terminal window while that tab is ACTIVE). Takes precedence over windowTitle.\n'run': Pane handle from key_locker launch_console — a decimal console hwnd, or the `wt:<pid>:<startMs>` form for a Windows Terminal tab. Targets THIS pane for the whole run+wait+read: the pane title is re-resolved before every read, so a mid-run retitle (e.g. a classic console renaming to `user@host` after an ssh login) is still tracked. Takes precedence over windowTitle. This is the paneId FIELD of launch_console's result — NOT its windowTitle. A Windows Terminal pane is tracked while its tab is the ACTIVE tab (switching away pauses the reads until you switch back)." - added
Input schema / properties / paneId / maxLengthAdded value: +48 - added
Input schema / properties / pasteKeyAdded value: +{ + "default": "auto", + "description": "Paste key combo. 'auto' picks ctrl+shift+v for WSL/bash/mintty/wezterm/alacritty, ctrl+v elsewhere. Only used when preferClipboard=true.", + "enum": [ + "auto", + "ctrl+v", + "ctrl+shift+v" + ], + "type": "string" +} - added
Input schema / properties / preferClipboardAdded value: +{ + "default": true, + "description": "Paste the input from the clipboard instead of typing it — safe for long text and not run through IME conversion (does not help while an IME composition is already in progress: the paste keystroke is consumed by the IME and nothing arrives, though the call still answers ok; hints.clipboard.imeOpen says when the IME was on at the paste). Your clipboard is replaced for the duration of the call and put back afterwards, but the restore is skipped when another process wrote to the clipboard first, when its contents are too large for an addon-less build to carry back, or when saving it failed; hints.clipboard reports which backend served the paste and whether the restore ran. Default true.", + "type": "boolean" +} - added
Input schema / properties / pressEnterAdded value: +{ + "default": true, + "description": "Press Enter after typing (default true).", + "type": "boolean" +} - added
Input schema / properties / readOptions / additionalPropertiesAdded value: +{} - added
Input schema / properties / readOptions / propertyNamesAdded value: +{ + "type": "string" +} - added
Input schema / properties / restoreFocusAdded value: +{ + "default": true, + "description": "Restore the previously-focused window after sending (default true).", + "type": "boolean" +} - added
Input schema / properties / sendOptions / additionalPropertiesAdded value: +{} - added
Input schema / properties / sendOptions / propertyNamesAdded value: +{ + "type": "string" +} - added
Input schema / properties / settleMsAdded value: +{ + "default": 300, + "description": "Milliseconds to wait after sending before checking foreground window (default 300).", + "maximum": 2000, + "minimum": 0, + "type": "integer" +} - added
Input schema / properties / sinceMarkerAdded value: +{ + "description": "Marker returned from a previous call. If found in current text, only the diff is returned.", + "maxLength": 64, + "type": "string" +} - added
Input schema / properties / sourceAdded value: +{ + "default": "auto", + "description": "'auto' = UIA TextPattern then OCR fallback; 'uia' = TextPattern only (fail on miss); 'ocr' = OCR only.", + "enum": [ + "auto", + "uia", + "ocr" + ], + "type": "string" +} - added
Input schema / properties / stripAnsiAdded value: +{ + "default": true, + "description": "Strip ANSI escape sequences (default true).", + "type": "boolean" +} - added
Input schema / properties / trackFocusAdded value: +{ + "default": true, + "description": "When true (default), detect if focus was stolen after sending. Reports focusLost in the response.", + "type": "boolean" +} - added
Input schema / properties / until / oneOfAdded value: +[ + { + "properties": { + "mode": { + "const": "quiet", + "type": "string" + }, + "quietMs": { + "default": 1500, + "description": "Stop when output is silent for this many ms (default 1500).", + "maximum": 30000, + "minimum": 50, + "type": "integer" + } + }, + "required": [ + "mode" + ], + "type": "object" + }, + { + "properties": { + "mode": { + "const": "pattern", + "type": "string" + }, + "pattern": { + "description": "Stop when output matches this string (or regex if regex:true)", + "type": "string" + }, + "quietMs": { + "description": "Optional settle fallback. When set, also completes with completion.reason:'quiet' (completion.matchedPattern stays absent — check it to tell a match from a settle) if output stays stable for this many ms WITHOUT the pattern matching, instead of waiting for the hard timeout. Use for commands that may finish without ever printing the pattern — e.g. a final line with no trailing newline that an end-anchored pattern (\\n / $) can't bind (issue #384). Omit to keep waiting for the pattern until timeoutMs.", + "maximum": 30000, + "minimum": 50, + "type": "integer" + }, + "regex": { + "default": false, + "description": "If true, treat pattern as a regex", + "type": "boolean" + } + }, + "required": [ + "mode", + "pattern" + ], + "type": "object" + }, + { + "properties": { + "mode": { + "const": "exit", + "type": "string" + }, + "shell": { + "default": "auto", + "description": "Shell the terminal runs, used to build the completion epilogue. 'bash' and 'powershell' are first-class. 'cmd' is not supported yet (returns ExitModeShellUnsupported). 'auto' (default) detects from the window process: a conhost-hosted shell resolves, but a nested/remote shell is invisible (an SSH/WSL session reports its OUTER process, so auto picks the outer shell and warns — pass shell explicitly for remote sessions); auto fails loudly (ExitModeShellAmbiguous) only for unknown hosts such as Windows Terminal. Inputs ending in an open construct (here-doc, unbalanced quote, unterminated $(…), here-string, trailing \\/backtick) are rejected with ExitModeUnsafeInput.", + "enum": [ + "bash", + "powershell", + "cmd", + "auto" + ], + "type": "string" + } + }, + "required": [ + "mode" + ], + "type": "object" + } +] - removed
Input schema / properties / until / typeRemoved value: -"object" - changed
Input schema / properties / windowTitle / descriptionPrevious value: -"Partial title of the terminal window (e.g. 'PowerShell', 'pwsh', 'WindowsTerminal'). Provide windowTitle OR paneId (paneId takes precedence)."New value: +"'read' / 'run': Partial title of the terminal window (e.g. 'PowerShell', 'pwsh', 'WindowsTerminal'). Provide windowTitle OR paneId (paneId takes precedence).\n'send': Partial title of the terminal window. Provide windowTitle OR paneId (paneId takes precedence)."
- Changed
wait_until4 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / target / propertiesAdded value: +{ + "by": { + "enum": [ + "text", + "regex", + "role", + "ariaLabel" + ], + "type": "string" + }, + "elementName": { + "type": "string" + }, + "elementSelector": { + "type": "string" + }, + "fromHwnd": { + "type": "string" + }, + "pattern": { + "type": "string" + }, + "port": { + "type": "number" + }, + "regex": { + "type": "boolean" + }, + "scope": { + "type": "string" + }, + "tabId": { + "type": "string" + }, + "windowTitle": { + "type": "string" + } +} - added
Input schema / properties / target / typeAdded value: +"object"
- Changed
window_dock9 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / height / exclusiveMinimumAdded value: +0 - added
Input schema / properties / height / maximumAdded value: +9007199254740991 - added
Input schema / properties / margin / maximumAdded value: +9007199254740991 - added
Input schema / properties / monitorId / maximumAdded value: +9007199254740991 - changed
Input schema / properties / title / descriptionPrevious value: -"Partial window title (case-insensitive)"New value: +"'pin' / 'unpin': Partial window title (case-insensitive)\n'dock': Partial window title to dock (case-insensitive). Matches the first shown window containing this text (a minimised one counts; a hidden one — on another virtual desktop, or a minimised packaged app's content — does not). Example: 'Claude Code', 'メモ帳'." - added
Input schema / properties / width / exclusiveMinimumAdded value: +0 - added
Input schema / properties / width / maximumAdded value: +9007199254740991
- Changed
workspace_launch3 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / args / itemsAdded value: +{ + "maxLength": 1000, + "type": "string" +}
- Changed
workspace_snapshot4 fields changed- added
Input schema / $schemaAdded value: +"http://json-schema.org/draft-07/schema#" - removed
Input schema / additionalPropertiesRemoved value: -false - added
Input schema / properties / thumbnailMaxDimension / exclusiveMinimumAdded value: +0 - added
Input schema / properties / thumbnailMaxDimension / maximumAdded value: +9007199254740991
4 tool updates
v2.2.0- Changed
desktop_state2 fields changed- removed
Input schema / properties / port / defaultRemoved value: -9222 - changed
Input schema / properties / port / descriptionPrevious value: -"CDP port for includeDocument (default 9222)."New value: +"CDP port for includeDocument. Default: the port the tab named by tabId was opened on, else 9222."
- Changed
focus_window1 field changed- changed
Input schema / properties / title / descriptionPrevious value: -"Partial window title to search for (case-insensitive)"New value: +"Partial window title to search for (case-insensitive). A shown window (a minimised one counts) is taken before a hidden one with the same text; a hidden one — on another virtual desktop — only when nothing shown matches."
- Changed
keyboard1 field changed- changed
Input schema / properties / use_clipboard / descriptionPrevious value: -"If true, copy text to clipboard and paste with Ctrl+V instead of simulating keystrokes. Use this when typing URLs, paths, or ASCII text into apps with Japanese IME active — pasted text is not run through IME conversion. Note this does not help while an IME composition is already in progress: the paste keystroke is consumed by the IME and nothing is inserted, so commit or cancel the composition first. Your clipboard is replaced for the duration of the call and put back afterwards; hints.clipboard reports which backend served the paste and whether the restore ran. On builds without the native addon this path is capped at about 12000 characters and fails with code:'ClipboardWriteTooLargeForFallback' above it. Default false."New value: +"If true, copy text to clipboard and paste with Ctrl+V instead of simulating keystrokes. Use this when typing URLs, paths, or ASCII text into apps with Japanese IME active — pasted text is not run through IME conversion. Note this does not help while an IME composition is already in progress: the paste keystroke is consumed by the IME and nothing is inserted, though the call still answers ok, so commit or cancel the composition first; when the IME was on at the paste, hints.clipboard.imeOpen says so. Your clipboard is replaced for the duration of the call and put back afterwards; hints.clipboard reports which backend served the paste and whether the restore ran. On builds without the native addon this path is capped at about 12000 characters and fails with code:'ClipboardWriteTooLargeForFallback' above it. Default false."
- Changed
screenshot2 fields changed- changed
Input schema / properties / dotByDot / descriptionPrevious value: -"1:1 pixel mode — no scaling, WebP compression. Window captures include 'origin: (x,y)' so you can compute screen position: screen_x = origin_x + image_x. When dotByDotMaxDimension is also set, scale factor is included: screen_x = origin_x + image_x / scale."New value: +"1:1 pixel mode — no scaling, WebP compression. Window captures include 'origin: (x,y)' so you can compute screen position: screen_x = origin_x + image_x. A frame from the window's composition surface (WGC: mode='background' with fullContent on a visible, non-minimised window when it can be, or normal mode's rescue) starts at the window's visible frame, and its origin is that frame's corner; when the visible frame cannot be read, no origin is given — the text says so; use desktop_discover for coordinates. When dotByDotMaxDimension is also set, scale factor is included: screen_x = origin_x + image_x / scale." - changed
Input schema / properties / mode / descriptionPrevious value: -"Capture mode.\n 'normal' — default. Window-targeted captures (windowTitle / hwnd) use Win32 PrintWindow with automatic BitBlt fallback when PrintWindow returns no data or an all-black frame; the route used is reported in hints.captureSource. Fullscreen / displayId captures use BitBlt.\n 'background' — explicit Win32 PrintWindow capture, retained for back-compat and explicit selection. Requires windowTitle (or hwnd). Pair with fullContent for GPU-rendered apps."New value: +"Capture mode.\n 'normal' — default. Window-targeted captures (windowTitle / hwnd) use Win32 PrintWindow with automatic BitBlt fallback when PrintWindow returns no data or an all-black frame; the route used is reported in hints.captureSource. Fullscreen / displayId captures use BitBlt.\n 'background' — a capture of the window itself, not of the screen: its composition surface (WGC) when it can be for a visible, non-minimised window with fullContent, else Win32 PrintWindow. Requires windowTitle (or hwnd). Pair with fullContent for GPU-rendered apps."
7 tool updates
v2.0.0- Changed
browser_click1 field changed- changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported."New value: +"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."
- Changed
browser_navigate1 field changed- changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported."New value: +"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."
- Changed
click_element1 field changed- changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported."New value: +"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."
- Changed
keyboard1 field changed- changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported."New value: +"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."
- Changed
mouse_click1 field changed- changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported."New value: +"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."
- Changed
mouse_drag1 field changed- changed
Input schema / properties / narrate / descriptionPrevious value: -"Narration level. rich includes UIA or browser state diff when supported."New value: +"Narration level. rich includes UIA or browser state diff when supported, and is withheld with post.rich.diffDegraded when the diff cannot be shown to describe the window that was acted on."
- Changed
scroll4 fields changed- changed
Input schema / properties / amount / descriptionPrevious value: -"Number of scroll notches (default 3). UIA-capable apps (Notepad, Explorer, WPF — Tier 1): empirically ≈1 text line per notch; amount:3 (default) ≈ 3 lines (small nudge), amount:10 ≈ 10 lines (~½ visible area). Legacy apps (SendInput path): each amount unit sends 3 wheel ticks; at Windows default 3 lines/tick that is ≈9 text lines per unit — distance varies by app/OS wheel-speed settings."New value: +"Number of scroll notches (default 3, max 1000). One notch = one physical wheel detent on every dispatch path. Count screenfuls, not lines: ≈10 notches move one screenful, so amount:3 (default) is about a third of a screen. UIA-capable apps (Notepad, Explorer, WPF) step ≈1/10 of the window's visible area per notch; browsers and WebView-based apps (Chrome, Electron, Tauri) move ≈100 px per notch. Exact distance depends on the app, the window size and the OS wheel-speed setting. The 1000-notch ceiling exists because each notch is dispatched as real wheel input; to reach a specific place in a long document use action='to_element' or action='smart' instead of a huge amount." - added
Input schema / properties / amount / maximumAdded value: +1000 - added
Input schema / properties / amount / minimumAdded value: +1 - added
Input schema / properties / target / minLengthAdded value: +1
1 tool update
v1.14.3- Changed
keyboard1 field changed- changed
Input schema / properties / use_clipboard / descriptionPrevious value: -"If true, copy text to clipboard and paste with Ctrl+V instead of simulating keystrokes. Use this when typing URLs, paths, or ASCII text into apps with Japanese IME active — prevents IME from converting characters. Default false."New value: +"If true, copy text to clipboard and paste with Ctrl+V instead of simulating keystrokes. Use this when typing URLs, paths, or ASCII text into apps with Japanese IME active — pasted text is not run through IME conversion. Note this does not help while an IME composition is already in progress: the paste keystroke is consumed by the IME and nothing is inserted, so commit or cancel the composition first. Your clipboard is replaced for the duration of the call and put back afterwards; hints.clipboard reports which backend served the paste and whether the restore ran. On builds without the native addon this path is capped at about 12000 characters and fails with code:'ClipboardWriteTooLargeForFallback' above it. Default false."
1 tool update
v1.13.0- Changed
terminal2 fields changed- added
Input schema / properties / paneIdAdded value: +{ + "type": "string" +} - changed
Input schema / properties / windowTitle / descriptionPrevious value: -"Partial title of the terminal window (e.g. 'PowerShell', 'pwsh', 'WindowsTerminal')."New value: +"Partial title of the terminal window (e.g. 'PowerShell', 'pwsh', 'WindowsTerminal'). Provide windowTitle OR paneId (paneId takes precedence)."
20 tool updates
v1.12.0- Added
browser_click - Added
browser_eval - Added
browser_fill - Added
browser_form - Added
browser_locate - Added
browser_navigate - Added
browser_open - Added
browser_overview - Added
browser_search - Added
click_element - Added
clipboard - Added
desktop_state - Added
excel - Added
focus_window - Added
key_locker - Added
keyboard - Added
mouse_click - Changed
screenshot2 fields changed- changed
Input schema / properties / confirmImage / descriptionPrevious value: -"Must be true to receive image pixels when detail='image'. Without this flag, detail='image' is blocked and a guidance message is returned instead. Prefer detail='text' / diffMode=true / dotByDot=true first — only set confirmImage=true when visual inspection is genuinely required."New value: +"Embed inline image pixels in the response. detail='image' now returns a cheap by-ref resource_link WITHOUT this flag (it is no longer blocked); confirmImage=true ADDITIONALLY embeds the inline image for immediate vision. detail='som' likewise returns its elements[] + a by-ref resource_link by default; confirmImage=true ADDITIONALLY inlines the annotated SoM bitmap. Prefer detail='text' / diffMode=true / dotByDot=true first — set confirmImage=true only when inline visual inspection is genuinely required." - changed
Input schema / properties / detail / descriptionPrevious value: -"Response detail level (omit to let the server pick a smart default):\n omitted — auto: 'image' when dotByDot/region/displayId is specified, else 'meta'\n 'meta' — window title + screen region only (~20 tok/window, cheapest)\n 'text' — UIA element tree as JSON with text values (~100-300 tok/window, no image)\n 'image' — actual screenshot pixels. BLOCKED unless confirmImage=true is also passed.\n 'som' — Set-of-Marks image + OCR elements (bypasses UIA entirely). BLOCKED unless confirmImage=true is also passed.\n 'ocr' — Windows OCR words with screen-pixel clickAt coords (Phase 4: absorbs former screenshot_ocr). Use when UIA returns no actionable elements (WinUI3 custom-drawn UIs, game overlays, PDF viewers). Note: detail='text' auto-falls back to OCR via ocrFallback='auto'; choose detail='ocr' only when forcing OCR unconditionally."New value: +"Response detail level (omit to let the server pick a smart default):\n omitted — auto: 'image' when dotByDot/region/displayId is specified, else 'meta'\n 'meta' — window title + screen region only (~20 tok/window, cheapest)\n 'text' — UIA element tree as JSON with text values (~100-300 tok/window, no image)\n 'image' — actual screenshot pixels. Returns a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to ALSO embed the inline image.\n 'som' — Set-of-Marks elements + annotated image (bypasses UIA entirely). Returns the OCR elements[] plus a cheap by-ref resource_link by default (no inline base64); pass confirmImage=true to ALSO embed the annotated bitmap.\n 'ocr' — Windows OCR words with screen-pixel clickAt coords (Phase 4: absorbs former screenshot_ocr). Use when UIA returns no actionable elements (WinUI3 custom-drawn UIs, game overlays, PDF viewers). Note: detail='text' auto-falls back to OCR via ocrFallback='auto'; choose detail='ocr' only when forcing OCR unconditionally."
- Added
screenshot_gc - Added
screenshot_query
16 tool updates
v1.10.4- Removed
browser_click - Removed
browser_eval - Removed
browser_fill - Removed
browser_form - Removed
browser_locate - Removed
browser_navigate - Removed
browser_open - Removed
browser_overview - Removed
browser_search - Removed
click_element - Removed
clipboard - Removed
desktop_state - Removed
excel - Removed
focus_window - Removed
keyboard - Removed
mouse_click
1 tool update
v1.10.3- Changed
screenshot2 fields changed- removed
Input schema / properties / ocrLanguage / defaultRemoved value: -"ja" - changed
Input schema / properties / ocrLanguage / descriptionPrevious value: -"BCP-47 language tag for the OCR engine (e.g. 'ja', 'en-US'). Used when detail='text' (OCR fallback) or detail='ocr' (direct OCR)."New value: +"BCP-47 language tag for the OCR engine (e.g. 'ja', 'en-US'). Auto-detects from system locale when omitted. Used when detail='text' (OCR fallback) or detail='ocr' (direct OCR)."
27 tool updates
v1.9.2- Added
browser_click - Added
browser_eval - Added
browser_fill - Added
browser_form - Added
browser_locate - Added
browser_navigate - Added
browser_open - Added
browser_overview - Added
browser_search - Added
click_element - Added
clipboard - Added
desktop_state - Added
excel - Added
focus_window - Added
keyboard - Added
mouse_click - Added
mouse_drag - Added
notification_show - Added
run_macro - Added
screenshot - Added
scroll - Added
server_status - Added
terminal - Added
wait_until - Added
window_dock - Added
workspace_launch - Added
workspace_snapshot
27 tool updates
v1.8.0- Removed
browser_click - Removed
browser_eval - Removed
browser_fill - Removed
browser_form - Removed
browser_locate - Removed
browser_navigate - Removed
browser_open - Removed
browser_overview - Removed
browser_search - Removed
click_element - Removed
clipboard - Removed
desktop_state - Removed
excel - Removed
focus_window - Removed
keyboard - Removed
mouse_click - Removed
mouse_drag - Removed
notification_show - Removed
run_macro - Removed
screenshot - Removed
scroll - Removed
server_status - Removed
terminal - Removed
wait_until - Removed
window_dock - Removed
workspace_launch - Removed
workspace_snapshot
TDQS
Scored across 30 tools
Each browser_* tool has a distinct purpose (open/click/fill/eval/overview/search/navigate/locate) and native tools split cleanly along click/keyboard/mouse/scroll/focus lines. However there is real overlap between browser_click vs browser_locate, mouse_click vs click_element, and screenshot vs workspace_snapshot vs desktop_state, and several tools (click_element, keyboard, desktop_state) lean on desktop_discover/desktop_act that aren't in the surface, muddying boundaries.
The browser_* and mouse_* families use a consistent namespace_verb pattern, but the rest mixes verb_noun (click_element, focus_window, run_macro, wait_until, scroll) with noun_verb (window_dock, workspace_launch, notification_show) and bare nouns (keyboard, clipboard, terminal, screenshot, excel). Names remain readable, but no single predictable convention governs the set.
30 tools is heavy and exceeds the typical 3-15 sweet spot, but the domain genuinely spans browser automation, native desktop control, terminal, Excel VBA, clipboard, and credential management, so most tools earn their place. Still, several are marginal (server_status, screenshot_gc, notification_show) making the surface feel padded.
The surface covers a broad lifecycle (navigate, act, observe, wait, snapshot, macro orchestration) with few obvious domain gaps. The notable hole is that descriptions repeatedly instruct agents to call desktop_discover/desktop_act for UIA discovery and value-setting, yet neither tool appears in the set, creating dead-end workflows.
Maintenance
Related MCP Connectors
LLM Orchestration Agent (Mcp)
Official Browserless MCP server: browser automation, scraping, screenshots and PDFs for AI agents.
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to interact with yo…
MCP server for visual regression testing: triage a PR's UI diffs from your coding agent.
Related MCP Servers
- AlicenseNot gradedqualityCmaintenanceGUI automation MCP server that enables AI agents to see and control the Windows desktop using a local Vision LLM (Ollama), supporting screenshot analysis, mouse/keyboard actions, and autonomous task execution.4MIT
- AlicenseNot gradedqualityAmaintenanceA Windows computer use agent — FastMCP server that gives AI assistants hands on the real desktop: windows, UI elements, mouse, keyboard, screenshots, OCR, shortcuts, dialogs, and outcome verification.44MIT
- AlicenseAqualityAmaintenanceAllow AI agents to see and control a real Windows PC you own: observe (UIA + screenshots), click/type/drag/scroll, launch apps, owner Live View. BYOH — your machine, your key.16Apache 2.0
- FlicenseAqualityAmaintenanceCross-platform desktop automation MCP server that lets AI agents capture screenshots, run OCR with UI-element classification, control mouse/keyboard, and launch programs on Linux, macOS, and Windows.201-