An MCP server for Windows computer use that lets agents observe and drive the desktop via UI Automation and fallback input, with human-coexistence safeguards.
Observe read-only: list/find windows, get window rects, list displays, cursor position, capture DPI-aware screenshots, read clipboard, identify UI elements at a point, read elements by ref, get a compact text tree (skyshot) with diffing, find elements by role/name, and check coexistence status.
Act via UI Automation: perform element actions (press, set_value, select, toggle, expand, collapse, scroll_into_view, focus) with refs or skyshot indices — no focus steal, no z-order dependence.
Send input: type text via targeted PostMessage, send keyboard shortcuts, click at client-area coordinates (element path first, raw cursor fallback).
Mutate desktop state: write clipboard, open applications/files/URIs with the shell.
Coexistence and safety: cross-agent serialization, refusal when the human is using the machine for physical input, and receipts distinguishing action accepted vs effect verified vs human/arbiter conflicts.
Cross-platform enumeration: the server starts and lists all 19 tools on non-Windows systems, but tool calls require Windows.
English · 中文
An MCP server + agent skill for computer use on Windows: accessibility element actions come first and screenshots are only the fallback. It ships a cross-session arbiter — when several agents share one machine it serializes them, and it yields while you are actually using the computer yourself.
This repository contains only the MCP server and the skill. It stays neutral toward any stdio MCP client (dsh / Claude Code / Codex / Cursor / Cline / ZCode …) — nothing here requires dsh.
Platform semantics (0.3.1 and later): the tools only work on Windows — they drive
user32/kernel32 and UI Automation. But the package also imports elsewhere and the server
starts there, answering tools/list as usual, so any client or directory crawler can
enumerate all 19 tools with their full schemas; actually calling a tool returns a clear
"requires Windows" error rather than the process failing to start at all. In 0.3.0 the import
itself raised, which made such crawlers unable to see the server at all
(tests/linux-handshake.py is the regression test for this property, and CI runs it on
ubuntu-latest).
A stdio MCP server exposing 19 tools. Every tool name is prefixed tool_, exactly as
tools/list returns it.
Observe (read-only, callable at any time): tool_skyshot (reads a window as a compact,
diffable text tree — three orders of magnitude smaller than a screenshot), tool_element_at_point,
tool_read_element, tool_find_elements, tool_capture_window (DPI-aware, cropped to the client area),
tool_list_windows / tool_find_window / tool_get_window_rect, tool_list_displays, tool_cursor_position,
tool_clipboard_read, tool_coexistence_status
Element actions (soft gate: serialized across agents, no physical input injected):
tool_element_action / tool_element_action_at — press / set_value / select / toggle / expand /
collapse / scroll_into_view / focus, delivered straight to the UIA element, so they
never steal focus and never care about z-order
Other mutating calls (soft gate too: the mutex, but no human-contention yield):
tool_type_text (targeted PostMessage), tool_clipboard_write, tool_open_application. They
synthesize no physical input, so they do not wait for you to stop working — and a clipboard
write still destroys whatever you last copied, so announce it when you do.
Physical input (hard gate: serialized across agents and yields to the human): exactly two
things share your one cursor and one keyboard — tool_click_at's raw_event path and
tool_send_keys' global hotkeys. The gate waits for the machine to go input-quiet, then refuses
with user-active rather than fight you for the cursor. tool_click_at tries its element path
first (ax_press), which injects no physical input and therefore takes the mutex only; the
receipt's method field says which path actually ran.
"Read-only" here means it takes no mutating action and synthesizes no input, so it is safe to
call while someone is using the machine. Two of them have a side effect worth knowing:
tool_capture_window writes the screenshot to disk (save_path; a temp file when omitted), and
tool_skyshot updates the server-side diff baseline it diffs the next shot against.
Every action returns a receipt rather than a self-reported success: action_sent /
effect_verified / foreground_changed / user-active / arbiter-busy. "The call was
accepted" and "the effect happened" are two different things, and the tool separates them for
the agent.
Related MCP server: openowl
There are already several mature open-source Windows implementations. dsh-cua's differences are concentrated on one thing: sharing a machine with a human.
dsh-cua | ||||
Element actions delivered as UIA patterns (no focus steal, z-order irrelevant) | ✅ | ✅ (ax mode) | ❌ reads via UIA, acts by coordinate click | via cua-driver |
Recent human input → refuse | ✅ | ❌ | ❌ | ❌ |
Cross-agent serialization (multi-process) | ✅ named mutex | ❌ | ❌ | ❌ |
Per-action effect assertion | ✅ three-state | reports a delivery tier | ❌ | ❌ |
Foreground-steal side effect measured | ✅ | ❌ | ❌ | ❌ |
Tool count | 19 | 59 | 15 | 6 |
The key distinction is two things that are routinely conflated:
"No focus steal" is a mechanism guarantee — either a UIA pattern or a targeted
PostMessage, so the cursor and keyboard focus are physically never touched. cua-driver has
it (ax mode). ahk-mcp does not, and the distinction is narrower than "no UIA": it reads
through UIA (ahk_uia_tree / ahk_uia_find / ahk_uia_url), but it has no UIA pattern
action — per its README it acts with coordinate clicks or synthetic keys, so an action does
move the real cursor.
"Yield the moment you move" is a timing guarantee — it reads the age of the human's last
input via GetLastInputInfo, waits when it sees you using the machine, and on timeout
refuses (user-active) instead of barging in. As of 2026-09-25 a pattern search across
the other three codebases in that table found no equivalent — that is search evidence, not
proof, and it covers input-age detection only: cua-driver does have human-facing guards of a
different kind (a consent requirement, and foreground-steal detection with restore).
effect_verified is likewise something the alternatives lack: it splits "the call was accepted"
from "the effect happened" and gives three states (true changed as expected / false accepted
but unchanged, downgraded to a failure / null no comparable state, i.e. unconfirmed). The
usual alternative is to re-observe once after the action and leave the judgement to the model.
What dsh-cua does not do (stated up front to avoid misunderstanding): no grounding of its
own — the server does not analyse pixels, so a text-only model cannot drive interfaces that
a tree cannot express (canvas, games, remote desktop). With a vision-capable model the pixel
path is supported end to end: capture_window returns the image together with a verified
image→screen mapping (bounds, scale, dpi_verified), and the model supplies the grounding.
Also not provided: record-and-replay, and an isolation sandbox. There are better-suited tools
for those.
Why not run the agent on a second desktop or a virtual display, so it never touches mine?
Because Windows has nothing to build that on, and the mobile design that does work rests on
exactly the missing piece. On Android an app can create a VirtualDisplay and address input at
it — an input event carries a display id, so the agent's taps are routed to its own screen and
the human's touchscreen never notices. Windows routes input per desktop, not per display: a
desktop has one input queue and one cursor position.
That makes the obvious analogues dead ends:
Idea | Why it does not isolate |
Add a virtual monitor (an indirect display driver) | Another canvas, not another cursor — the pointer still has a single position across all monitors |
A Windows virtual desktop ( | A view switch inside the same session: same input queue, same cursor |
A second session (RDP, or another user) | Isolation is real, but client Windows allows one interactive session per user at a time — connecting remotely locks the console, so the human loses their screen, which was the whole point |
A hidden Win32 desktop ( | The agent would get its own input queue and cursor, but its windows are invisible, so it can only drive instances it launched — not the program you are looking at. (Inferred from the window-station/desktop model; not measured here.) |
So the conflict is not a gap in this implementation, it is an OS constraint: Windows has no second cursor. Given that, yielding is the only correct response — and most calls never get near the problem, because they inject no input at all:
element_action / element_action_at — a UIA pattern is delivered to the element: no physical
input, no cursor movement, no focus change. These are the paths that "can run while the user
types".
type_text — a window-targeted PostMessage, not global keystrokes.
Every read-only tool (skyshot, element_at_point, capture_window, …) — touches nothing.
Exactly two paths inject physical input, and they exist because canvas-, game- and
Chromium-internal surfaces expose no element to address: the raw-event path of tool_click_at,
and the global hotkeys of tool_send_keys. Those two are what the arbiter guards.
You need Windows x64 + an interactive desktop session + Python ≥3.10 to actually drive a
desktop. (The package installs and starts on Linux/macOS too, tools/list answers normally,
and a tool call then reports "requires Windows" — see "platform semantics" above.)
# Option 1: uvx, zero install (recommended)
uvx dsh-cua # runs the stdio MCP server directly
# Option 2: pip
pip install dsh-cua
# Option 3: from source
pip install git+https://github.com/Hutusion/dsh-cua.gitHowever you install it, start the server with python -m dsh_cua:
python -m dsh_cua # depends on no executable being on PATHWhy the README does not say
dsh-cua-server: pip installs console scripts into the interpreter'sScriptsdirectory, and that directory is not necessarily on PATH — measured on a stock python.org 3.12 install, neither the User nor the Machine PATH contained it, sopip install dsh-cuasucceeded whiledsh-cua-serverreported command not found.python -mneeds no PATH entry at all. The console script is still shipped and works when PATH does contain it.Options 1 and 2 both work today: the package is published on PyPI (https://pypi.org/project/dsh-cua/). If
uvx/pipever 404s, use option 3 — it always works.
Any MCP client; name the server win32 (the skill's tool-name convention is
mcp__win32__*).
python -m (no PATH dependency, recommended):
{ "mcpServers": { "win32": { "command": "python", "args": ["-m", "dsh_cua"] } } }uvx:
{ "mcpServers": { "win32": { "command": "uvx", "args": ["dsh-cua"] } } }More shapes are in examples/: Claude Code / generic clients / a dsh
cordis.patch.yml fragment / the route modality declaration you need if you want the model to
read screenshots (tr-route-settings.yml).
skill/computer-use/SKILL.md is the companion doctrine for using
these tools: the observe → locate → act → verify loop, receipt semantics, retry safety, and the
discipline of coexisting with a human. The model can use the tools without it, but with it the
model picks the right path by itself — the measured difference is large. Copy it into your
skills directory:
# Claude Code / generic agents
cp -r skill/computer-use ~/.agents/skills/
# dsh
cp -r skill/computer-use ~/.dsh/skills/Tier | Operations | Gate |
Read-only | the 12 observe tools | no gate, callable at any time |
Soft | element actions, PostMessage typing, clipboard write, launching applications | cross-agent mutex (named mutex, multi-process, automatic serialization) |
Hard | raw clicks, global hotkeys | mutex + |
Honest boundaries: yielding is a cooperation protocol, not a hard guarantee (the tight check
150 ms before injection narrows the window as much as possible); a few applications
self-activate even on set_value (the receipt reports foreground_changed truthfully); and
two operators on the same window has no technical solution — do not drive the same window the
agent is driving.
python tests/verify-coexistence.py # 25 checks: zero-input proof / cross-process mutex / synthetic human contention / kill switch
python tests/verify-p0-fixes.py # the three P0s fixed in 0.2.0: each fails before the fixThe tests need no human cooperation — "user input" is synthesized with one real 1-pixel cursor move, and the cursor is restored afterwards.
The tests need a real interactive desktop session (some checks create windows and address
them through UIA), so they cannot run on a GitHub-hosted runner. What CI does cover is the
part that needs no desktop: packaging and installation, module import, regressions for the diff
index and tree-line escaping, and the arbiter's decision logic — see
.github/workflows/ci.yml.
python tests/ci-desktop-free.py # the local equivalent of the above, no desktop neededMIT
Capture a window screenshot and SAVE IT TO A FILE. Provide hwnd or title substring. Crops to the window's CLIENT area by default, so image pixel (0,0) is the same point click_at calls (0,0) — no frame offset to guess. Returns bounds = the screen rect (x,y,width,height) the image maps to, plus scale (multiply image pixels by it to get screen pixels; 1.0 unless you asked for downscaling). Also reports dpi_verified; when false, image pixels and input coordinates cannot be safely paired. Defaults to JPEG q80 — a 1080p shot is ~6-10x smaller than PNG, which is what keeps a long session's request body under the provider's 32 MiB cap. The bytes are NOT inlined as base64 by default (that costs ~12,500 tokens per shot as text); read the saved_path with read_image instead, which attaches the same picture for ~50 tokens. Pass include_data_uri=true only when a caller genuinely cannot read a file. Read-only — no input injection, no focus change.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| title | No | ||
| max_dim | No | ||
| quality | No | ||
| save_path | No | ||
| client_only | No | ||
| image_format | No | jpeg | |
| include_data_uri | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure — and it goes far beyond basic mutation info. It discloses read-only status ('no input injection, no focus change'), explains the JPEG-vs-PNG size tradeoff relative to the 32 MiB provider cap, quantifies token costs (~12,500 vs ~50 tokens), and warns about the dpi_verified=false pairing hazard. This is exemplary behavioral transparency with zero annotation support.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every clause earns its place — the coordinate alignment, file/scale/bounds return contract, size rationale, and token economics all carry operational value. It's front-loaded with the primary purpose and cropped-client-area detail. Slightly essay-like ('which is what keeps a long session's request body under the provider's 32 MiB cap') but purposeful; each sentence advances understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 8-parameter tool with no annotations and zero schema coverage, this description is remarkably complete: it explains the return contract (bounds, scale, dpi_verified) even though an output schema exists, covers file-saving semantics, and anticipates the exact failure mode an agent would hit (huge base64 payloads). Nothing critical an agent needs to invoke it correctly or interpret its output is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate — and it does, covering nearly every parameter in prose: hwnd/title (provide either), client_only (client area cropping), max_dim (downscaling), quality (JPEG q80 default), include_data_uri (token cost and base64 inlining), and save_path (read with read_image). A minor gap is that image_format is never explicitly named, and some parameters aren't labeled by their schema property names, but the intent is conveyed well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Capture a window screenshot and SAVE IT TO A FILE') and adds a distinguishing detail — cropping to the client area with aligned coordinate systems — that differentiates it from siblings like tool_skyshot. The purpose is unambiguous and the resource (window) is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit input guidance ('Provide hwnd or title substring') and a conditional rule for when to pass include_data_uri ('only when a caller genuinely cannot read a file'), plus a recommended alternative (read_image). However, it never names sibling tools as explicit exclusions (e.g., when to prefer tool_skyshot or tool_get_window_rect over this), so it lacks direct when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Click at client-area coordinates within a window. Coordinates relative to window content area (0,0 = top-left). Tries the ELEMENT path first (resolve the point via UI Automation and invoke that element) because it needs no focus and does not care about z-order; falls back to a raw cursor click, which does. The result's method says which was used: 'ax_press' or 'raw_event'. MUTATING: serialized across agent sessions; the raw_event path ALSO YIELDS TO THE USER (waits for input-quiet, then refuses with user-active rather than fighting them for the cursor); the ax_press path injects no physical input. The raw path verifies the addressed window actually owns that screen point and returns success=false instead of clicking another window. Pass dry_run=true to check where the click would land without moving the cursor, clicking, or taking any gate.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| hwnd | No | ||
| title | No | ||
| prefer | No | element | |
| dry_run | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It discloses mutation, serialization, user yielding for the raw path, z-order/focus dependencies, window ownership verification, and dry_run side-effect-free behavior. This is unusually rich and goes well beyond a simple 'click'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but front-loaded with the core purpose and every sentence adds a distinct behavioral fact. The MUTATING and dry_run sentences are dense but necessary; could be slightly tightened but overall well structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-param tool with no annotations and no schema descriptions, the description covers most behavioral context but leaves title and full prefer semantics undocumented. It is close to complete but not fully sufficient for all invocation decisions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description must document parameters. It explains x/y are client-area coordinates, dry_run prevents effects, prefer relates to element vs raw, and hwnd as 'addressed window'. Title is not documented, and prefer values beyond default 'element' are unstated, so compensation is strong but incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action ('Click at client-area coordinates within a window') and gives coordinate meaning. It explains internal click paths but does not explicitly differentiate from sibling tools like tool_element_action_at or tool_send_keys, so it lacks direct sibling contrast.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how the tool behaves (element first, raw fallback, dry_run) and when it is safe to use, but it never tells the agent which sibling to choose instead. No exclusions or comparisons to alternatives are given, so usage guidance is implied by the action itself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read the current clipboard text. Read-only. Non-text clipboard content reports empty text rather than failing.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses a key behavioral trait: non-text clipboard content reports empty text rather than failing. This goes beyond the basic read-only annotation and helps the agent anticipate edge cases. However, it does not mention potential side effects (e.g., clipboard access permissions) or whether the clipboard is cleared after reading, but for a read operation this is minor.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no wasted words. It front-loads the core action ('Read the current clipboard text'), then adds the read-only note and the edge-case behavior. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read tool, the description is nearly complete. It states what it does, the read-only nature, and the behavior for non-text content. The only minor gap is not describing the exact return format (e.g., whether it returns a string or an object), but the output schema exists and likely covers that. The description is sufficient for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description does not need to explain parameter semantics. The schema is empty and coverage is 100%, meaning there is nothing undocumented. The description adds value by clarifying the return behavior for non-text content, which is more useful than parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reads the current clipboard text, using a specific verb ('Read') and resource ('current clipboard text'). It also distinguishes itself from the sibling tool_clipboard_write by focusing on reading rather than writing. The behavior for non-text content is explicitly described, which further clarifies its purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: whenever an agent needs to retrieve the current clipboard text. It does not explicitly name alternatives or exclusions, but the sibling list includes tool_clipboard_write, and the read-only nature is clear. The context is sufficient for an agent to select this tool over writing to the clipboard.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Replace the clipboard contents with text. MUTATING (soft): serialized across agent sessions — and it DESTROYS whatever the user last copied, so do not use it casually; announce it when you do.
| Name | Required | Description | Default |
|---|---|---|---|
| text | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the behavioral burden. It explicitly labels the tool as MUTATING, warns that it DESTROYS the user's last copied content, mentions serialization across sessions, and instructs the agent to announce the action. This exceeds typical disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no fluff. The primary action is front-loaded, and the critical warnings are placed immediately after, making the most important information easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter write tool, the description covers the action, destructive consequences, persistence behavior, and usage caution. The presence of an output schema means return-value details are unnecessary here, so the description is complete for safe invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no descriptions for the single 'text' parameter, and the description only vaguely maps to it via 'with text.' This provides basic meaning but does not fully compensate for the 0% schema coverage, so it lands at an adequate but not exceptional level.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Replace the clipboard contents with text,' which is a specific verb and resource. It clearly identifies the tool as the write counterpart to clipboard_read and leaves no ambiguity about its function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises against casual use and instructs the agent to announce the action, giving clear context for safe usage. It does not explicitly name an alternative tool or state when not to use it beyond the strong caution, so it falls just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Report the coexistence policy in force: whether the admission gate is enabled, the quiet/max-wait/mutex-timeout tunables, the pre-injection final-check window, named-mutex health, and how long ago the last physical input happened. Read-only — call it when an action was refused with user-active or arbiter-busy and you want the numbers behind the refusal.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does disclose the key behavioral trait: 'Read-only.' It also sets expectations about the returned data by listing tunables, health, and timing information. It does not cover access prerequisites or failure behavior, but those are less critical for a stateless status report.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence with the core purpose front-loaded before a colon-delimited list of report contents. Every phrase earns its place, and the description avoids padding while remaining technically precise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only status tool with an output schema, the description provides both when to invoke it and what data to expect. The output schema covers the return structure, so nothing an agent needs to call this tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is no schema description to compensate for; the 0-param baseline is 4. The description still adds semantic context by enumerating what the caller receives, which is useful even though there are no inputs to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Report') and resource ('coexistence policy in force'), then enumerates exact contents such as the admission gate, tunables, named-mutex health, and last physical input. This clearly distinguishes it from sibling action tools like tool_click_at and tool_send_keys, which perform UI operations rather than report policy state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to call it: 'call it when an action was refused with user-active or arbiter-busy and you want the numbers behind the refusal.' It also frames the tool as read-only, making it clear this is a diagnostic fallback rather than a routine action; no alternative tool is needed because no sibling offers this state.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Current cursor position plus the screen size, both physical pixels. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'Read-only', indicating no side effects, and specifies that values are in physical pixels, which is critical for coordinate interpretation. This provides meaningful transparency beyond a bare 'returns position' statement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that conveys the primary purpose (cursor position and screen size) and a key qualifier (physical pixels), followed by the 'Read-only' trait. There is no wasted prose, and the most important information appears first.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (no parameters) and an output schema exists (not shown but indicated), so the description need not explain return types. It covers purpose, units, and side-effect nature. Minor details like multi-monitor behavior are not addressed, but for a basic read tool this is adequate. The presence of an output schema reduces the burden further.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4 per the rubric. The description adds no parameter semantics (there are none to explain), but it does clarify the output semantics (position and screen size), which is useful context. No gaps exist since there are no parameters to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns the current cursor position and screen size in physical pixels, which is a specific purpose distinct from sibling tools like tool_click_at (which manipulates coordinates) or tool_get_window_rect (which returns window bounds). The 'Read-only' phrase further disambiguates it as a query operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an agent needs the cursor position or screen dimensions, but it does not explicitly mention when to choose this tool over alternatives or list exclusions. Since the purpose is self-contained and no alternative is cited, the guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Perform an action on a UI element by the ref from element_at_point — this is the focus-free path: a UI Automation pattern is delivered to the element, so the window is never raised and z-order never matters. MUTATING (soft): serialized across agent sessions; NO physical input is injected, so it can run while the user types — just do not operate the very window the user is working in. action must be one of the actions listed for that element (press, set_value, select, toggle, expand, collapse, scroll_into_view, focus); set_value also needs text. Refuses with the supported list when the element does not offer the action.
| Name | Required | Description | Default |
|---|---|---|---|
| ref | Yes | ||
| text | No | ||
| action | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral burden and does so thoroughly. It discloses mutation ('MUTATING (soft)'), serialization across sessions, lack of physical input injection, the ability to run during user typing, and refusal behavior with the supported action list. This is far beyond the minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place. It front-loads the core purpose, then adds constraints, safety context, and parameter semantics in a logical order. No filler or tautology.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters and an output schema, the description is complete. It covers action constraints, the source of ref, the mutating behavior, safety implications, and refusal behavior. Return values are not explained, but the output schema already handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains ref as coming from element_at_point, enumerates valid action values, and explicitly ties text to set_value. This adds substantial meaning to all three parameters, including the optional text parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Perform an action'), a precise resource ('a UI element by the ref from element_at_point'), and a distinguishing property ('focus-free path'). The description clearly differentiates this from coordinate-based or physical-input tools like tool_click_at and tool_element_action_at.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: use this path when you have a ref from element_at_point and want a focus-free action. It also gives a concrete usage caveat ('do not operate the very window the user is working in') and notes it can run while the user types. It does not explicitly name alternatives or exclusions, but the guidance is strong enough to infer appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Act on the element that line index of the last skyshot named. This is the focus-free path for elements element_at_point cannot reach: the tree names the element, so the action goes to it and the window is never raised. MUTATING (soft): serialized across agent sessions; no physical input injected. action is one of press, set_value, select, toggle, expand, collapse, scroll_into_view, focus; set_value also needs text. Take a fresh skyshot if the UI has changed — a stale index is refused rather than acted on.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | Yes | ||
| text | No | ||
| index | Yes | ||
| action | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool is 'MUTATING (soft)', is serialized across agent sessions, injects no physical input, never raises the window, and refuses stale indexes rather than acting on them. These are all meaningful behavioral traits that an agent needs to know.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences pack in the core action, the differentiator, behavioral caveats, parameter options, and freshness guidance. Nothing is redundant, and the most important information (what the tool does and when to use it) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with four parameters and an output schema, the description covers the essential decision points: what index means, what actions are available, the soft-mutation semantics, and the stale-index safeguard. It doesn't detail every possible error or the return format, but the output schema presumably handles that. The description is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must explain the parameters. It does: it enumerates the valid action values, notes that set_value additionally requires text, and explains that index refers to a line in the last skyshot. It does not elaborate on hwnd or the exact meaning of each action, but the critical disambiguation is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Act on') and resource ('element that line index of the last skyshot'), and immediately distinguishes itself from a sibling tool by calling itself the 'focus-free path' for elements element_at_point cannot reach. The list of possible actions further clarifies its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names the sibling alternative (element_at_point) and gives a concrete condition for when to use this tool instead: 'for elements element_at_point cannot reach'. It also advises taking a fresh skyshot when the UI changes, which is a clear operational rule. No other alternatives are mentioned, but the primary decision point is covered.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Identify the UI element at a PHYSICAL screen point via Windows UI Automation. Returns its name, role, rect, and which actions it supports, plus an opaque ref for read_element. This is how you turn 'what I see in the screenshot' into something addressable without clicking: coordinates here are the same space as capture bounds (both physical pixels). Read-only — moves no cursor and changes no focus.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explicitly states 'Read-only — moves no cursor and changes no focus', which is a key behavioral disclosure. It also mentions the return of an opaque ref. It doesn't cover error conditions or edge cases like no element at the point, but for a read-only query the main behavioral trait is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, with no filler. The core purpose is front-loaded, followed by return values and the crucial coordinate-space clarification, and ends with a safety note. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has only two parameters and an output schema, so the description doesn't need to explain return values. It covers the use case, coordinate space, and read-only behavior. The only minor gap is not addressing the case where no element exists at the point, but the output schema likely handles that. Overall it is complete enough for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains that x and y are physical pixel coordinates in the same space as capture bounds, which is critical for correct usage. It also emphasizes 'PHYSICAL screen point' to avoid coordinate-space ambiguity. This adds meaning well beyond the bare integer type in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Identify') and resource ('UI element at a PHYSICAL screen point via Windows UI Automation'), then lists exactly what is returned (name, role, rect, supported actions, opaque ref). It also distinguishes itself from siblings by framing it as the way to turn a screenshot point into an addressable element without clicking, clearly separating it from click_at and read_element.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states when to use it: when you have a physical screen coordinate from a screenshot and want to address the element without clicking. It notes the coordinate space matches capture bounds. It does not explicitly name alternatives or state when not to use it, but the purpose is clear enough that an agent can infer it is the right tool for point-based element lookup.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Find UI elements by role and/or name substring and return refs that can be acted on with element_action. This is how you reach an element that cannot be hit-tested — the point of naming it is that you no longer need a pixel. role is the lowercase role from skyshot (button, edit, document, list_item, ...). At least one filter is required: an unfiltered call would be a tree dump, so use skyshot for that. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| role | No | ||
| title | No | ||
| max_results | No | ||
| automation_id | No | ||
| name_contains | No | ||
| include_offscreen | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the behavioral burden. It discloses read-only behavior, the need for at least one filter, and the consequence of an unfiltered call (tree dump). It does not mention edge cases like no matches or offscreen behavior, but the core safety and boundary characteristics are clearly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: it front-loads the primary action and output, then gives usage rationale, role semantics, filter requirement, and safety note. Every sentence contributes meaningful guidance with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter tool with no annotations, the description covers the core intent, the key role semantics, the mandatory filter condition, and the read-only nature. An output schema exists, so return-value details are not required. The main omission is fuller parameter-level guidance, but the description is sufficient for competent selection and basic correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the 'role' parameter (lowercase role from skyshot, with examples) and implies 'name_contains' means substring matching. However, it leaves hwnd, title, automation_id, max_results, and include_offscreen essentially unexplained, so the compensation is only partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource ('Find UI elements'), states the matching criteria ('by role and/or name substring'), and explains the output ('return refs that can be acted on with element_action'). It also distinguishes itself from pixel/hit-test based tools, so an agent can separate it from siblings like element_at_point and skyshot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use this tool: to reach an element that cannot be hit-tested, because naming it removes the need for a pixel. It also gives an exclusion: an unfiltered call would be a tree dump, so use skyshot for that. The 'at least one filter is required' rule further guides correct invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Find a window by title substring and return its hwnd and geometry. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| title | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the tool is read-only, which is a key behavioral trait, and states the return content (hwnd and geometry). However, it does not mention behavior when no match is found, case sensitivity, or whether multiple matches are possible. This is minimal but not absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that states purpose and read-only nature efficiently. It could include a bit more detail without losing conciseness, but it is appropriately brief and wastes no words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, return values are covered. The tool is simple with one parameter. Missing details include behavior on no match, multiple matches, and case sensitivity, which could affect an agent's interpretation. The description is adequate but not fully complete for all potential edge cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain the parameter. It states the 'title' is used for substring matching, which adds meaning beyond the schema's bare type and default. It does not clarify edge cases like empty string behavior, but the core semantic is conveyed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Find'), a specific resource ('window'), and the distinguishing mechanism ('by title substring'), and specifies the returned data ('hwnd and geometry'). This clearly differentiates it from sibling tools like tool_list_windows and tool_get_window_rect, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a clear use case: given a title substring, find a window. However, it does not explicitly mention alternatives or when not to use this tool (e.g., when listing all windows or retrieving geometry of an already known window). The usage guidance is adequate but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get detailed position and size info for a window: absolute position, dimensions, and client area size. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| title | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states 'Read-only,' which is a key safety trait, but it does not disclose other behaviors: what happens if the window is not found, whether hwnd or title is used preferentially, whether the window must be visible, or any error conditions. For a tool with two optional parameters, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the core purpose and includes the read-only safety note. Every word earns its place; there is no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the output schema exists to explain return values, the description does not cover essential context: how to choose between hwnd and title, what happens if neither is provided (since both have defaults), or whether the tool requires a valid window handle. For a tool with two optional parameters, this is incomplete and could lead to incorrect invocations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not explain the parameters at all. It does not clarify that hwnd and title are alternative identifiers, which one takes precedence, or how they interact. The description's mention of 'absolute position, dimensions, and client area size' describes the output, not the input. This leaves the agent guessing how to specify the target window.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: getting detailed position and size info for a window, including absolute position, dimensions, and client area size. This is a specific verb (get) and resource (window), and it distinguishes from siblings like tool_find_window (which likely finds handles) and tool_capture_window (which captures images). It is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites (e.g., need a window handle first) or scenarios where this should be preferred over tool_list_windows or tool_find_window. The description is purely declarative without any contextual direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List monitors: index (1-based), primary flag, device name, and the monitor plus work-area rectangles in physical pixels. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden. It explicitly says 'Read-only', which communicates non-mutating safety, and specifies the exact information returned, including the physical-pixel coordinate context. This is adequate for a simple list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence front-loads the action and resource, then lists the output fields in a readable sequence. Every phrase earns its place, and the read-only note is a useful signal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool takes no parameters, has an output schema, and the description adds semantic meaning about the returned items (work-area rectangles, physical pixels, primary flag). An agent has everything it needs to select and invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description does not need to explain parameter semantics. The base baseline of 4 applies; there is nothing missing and the description adds no irrelevant parameter information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List monitors') and resource ('monitors') and enumerates the exact fields returned (index, primary flag, device name, monitor and work-area rectangles in physical pixels). It clearly distinguishes from display-related siblings by scope and output details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance about when to use this tool versus alternatives like tool_get_window_rect or tool_cursor_position. The description implies a read-only enumeration use case but does not state exclusions or conditions that would route an agent to this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all visible top-level windows with title, hwnd, PID, position and size, sorted largest first. Read-only — safe while the user works.
| Name | Required | Description | Default |
|---|---|---|---|
| filter_title | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool is read-only and safe while the user works, and it discloses the sorting behavior and scope ('visible top-level windows'). This goes beyond the schema and gives an agent confidence about side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the action and resource, then packs in the key output fields and ordering. Every clause earns its place with no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema exists, so return values are presumably covered elsewhere. The description covers purpose, scope, sorting, and safety, but it omits filter_title semantics and does not provide guidance on when to prefer this over sibling tools. For a simple list tool this is acceptable but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, filter_title, is not mentioned in the description. Since schema description coverage is 0%, the description should explain what filtering means (e.g., substring match, case sensitivity), but it does not. The parameter name and default empty string imply optionality, but no semantic detail is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List'), a precise resource ('all visible top-level windows'), and the exact fields returned (title, hwnd, PID, position, size). It also mentions sorting behavior, which distinguishes it from sibling tools like tool_find_window or tool_get_window_rect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: when you need an overview of visible top-level windows. It also notes the operation is read-only and safe while the user works, which is useful context. However, it does not explicitly contrast this tool with alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Open a file, folder, or URI with the shell's default handler — like double-clicking it in Explorer: an .exe path, a document, or a URL. MUTATING (soft): serialized across agent sessions; the launched app may take focus.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it discloses important behavioral traits: it is labeled MUTATING (soft), serialized across agent sessions, and may take focus. This goes beyond a bare 'open' statement. The term 'soft' is somewhat informal, but the sentence adds genuinely useful side-effect context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with no filler: the first establishes the operation and examples, the second adds critical behavioral caveats. Every clause earns its place, and the key purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one required parameter) and has an output schema, so the description does not need to explain return values. It covers what is opened, how, and the relevant side effects (session serialization, focus stealing), making the definition complete for an agent deciding whether and how to call it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate for the single 'target' parameter. It does so by defining target as a file, folder, or URI and giving examples (.exe path, document, URL). It could add path-format or URI-scheme details, but the required meaning is clear enough for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Open') and a precise resource class ('a file, folder, or URI') via the shell's default handler, with concrete examples like .exe paths, documents, and URLs. This clearly distinguishes it from the UI-automation siblings, none of which launch external default-handler targets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear mental model — same as double-clicking in Explorer — so an agent knows to use it when it wants OS-level default handling rather than window/element manipulation. It does not explicitly name alternatives or exclusion conditions, but the sibling tools are sufficiently distinct that no close alternative needs to be ruled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Re-read a UI element by the ref returned from element_at_point. Use this to check whether an element is still there before acting on it; a ref goes stale when the UI is rebuilt, and that is reported rather than guessed. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| ref | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden. It discloses that the tool is read-only and explains that stale refs are reported rather than guessed. It doesn't detail error handling or output format, but these are likely covered by the output schema. The key behavioral trait of staleness reporting is clearly communicated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the core action and source stated first, followed by the usage scenario and staleness behavior. Every word earns its place; there is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read tool with an output schema (even though not shown), the description covers purpose, usage, and the key behavioral nuance of staleness. The agent has enough to decide when to call it and what to expect, and the output schema presumably covers return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains that 'ref' is a reference returned by element_at_point and clarifies that it can become stale. This adds meaning beyond the bare 'string' type, giving the agent enough context to use the parameter correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('re-read') and resource ('UI element'), and clarifies the ref comes from element_at_point. It distinguishes itself by focusing on re-reading a specific element rather than performing actions on it, making its purpose clear and unique among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit guidance on when to use it ('Use this to check whether an element is still there before acting on it') and explains the stale ref scenario. It doesn't name alternative tools explicitly, but the context implies its role in the verification step. A slight gap is the lack of explicit 'when not to use' guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Send keyboard shortcut to a window. Example: keys=['ctrl','s']. For Alt+key menu navigation, include 'alt' as first key (or set use_alt_key). MUTATING: serialized across agent sessions via the admission gate, and global hotkeys additionally YIELD TO THE USER — recent mouse/keyboard activity makes them wait, then refuse with reason user-active instead of interrupting; switch to element actions or retry later. Also refuses when the window is not the foreground window, because injected keys would go to the wrong window.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| keys | No | ||
| title | No | ||
| use_alt_key | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description carries the full burden and it delivers: it discloses that the tool is mutating, serialized via admission gate, that global hotkeys yield to the user and refuse with reason user-active, and that it refuses when the target window is not the foreground window. That is rich behavioral disclosure beyond a bare 'send keys' statement. A 5 would require even more, like what exactly happens on failure, but this is substantial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: the example, the Alt-navigation tip, the serialization/yield-to-user behavior, and the foreground-window precondition. No filler and information-dense.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a 4-parameter tool with no annotations but a present output schema and 18 siblings, the description is complete enough to invoke correctly: window targeting (hwnd/title), shortcut syntax, Alt-key mode, and two refusal conditions are all covered. The output schema exists to explain return values, so that is not a gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, and it does for the key parameters: keys get the example ['ctrl','s'], use_alt_key gets the 'alt as first key' guidance, and title/hwnd are the window-targeting pair. It doesn't explain the hwnd=0 default or the title fallback relationship explicitly, but it covers the two non-obvious parameters. Baseline would be 3 at 100% coverage; with 0% coverage and this much guidance, a 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: send keyboard shortcut to a window. Not fully differentiated from sibling tool_type_text, but it names the target and the key-list mechanism clearly, and the title is null so the description carries the load.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives when-to-use guidance: use for shortcuts like ctrl+s, use 'alt' first key for menu navigation. It also gives a when-not-to-use and an alternative: switch to element actions, or retry later, when user is active. This clearly routes the agent compared with siblings like tool_click_at, tool_type_text, and tool_element_action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read a window's UI as a compact TEXT tree instead of a screenshot — far cheaper than an image, and it names elements a hit-test cannot reach (e.g. an input inside a Chromium page, which element_at_point reports only as the enclosing 'document'). Each line is '{index} {indent}{role} {name}{Value: ...}{(state)}'. Every shot after the first is a DIFF against the previous one: unchanged lines are omitted, '' marks a changed line, '+' an added one, and removed indices are summarised as ranges. So a second call is usually a few lines, not the whole tree — an unchanged window costs about 100 characters. INDEXES ARE POSITIONAL: each shot numbers the tree it just walked, so a number is only valid against the shot that printed it. '' and '+' lines carry THIS shot's index and may be acted on; unchanged lines carry no index, so reaching one needs disable_diff=true for a full render. An index read from an older shot is not a durable handle — re-check it with a fresh shot before acting if the window may have changed. The result includes shot_key (the window the diff baseline belongs to — the baseline lives server-side and outlives your session, so this is how you confirm it was taken against the window you meant). Pass include_offscreen=true to include elements that are scrolled out or hidden. NOTE: this reads element VALUES, so the text can contain whatever is on screen in that window. Read-only — no input injection, no focus change; safe to call while the user works.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| title | No | ||
| max_depth | No | ||
| max_nodes | No | ||
| disable_diff | No | ||
| include_values | No | ||
| include_offscreen | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the behavioral burden, and it does so exceptionally. It explains the diff behavior between shots, the exact line format, positional index caveats, the server-side shot_key baseline, offscreen inclusion, and that values may contain arbitrary on-screen text. It also explicitly declares the tool is read-only with no input injection or focus change, which is critical behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but densely packed with necessary operational details. The core purpose is front-loaded, followed by output format, diff semantics, index validity, and safety. A few details could be tightened, but the length is justified by the tool's complexity and the absence of annotation support.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with no annotations, the description covers nearly everything an agent needs: output format, diff semantics, positional index pitfalls, shot_key behavior, offscreen handling, and read-only safety. It is slightly incomplete on how to target the window via hwnd or title and does not clarify max_depth or max_nodes, but the overall picture is very strong.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does explain disable_diff, include_offscreen, and indirectly include_values through the note about reading element values. However, it does not explain hwnd, title, max_depth, or max_nodes, although their names and defaults are somewhat self-evident. This is partial compensation, not full, so a 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action and resource: 'Read a window's UI as a compact TEXT tree instead of a screenshot'. It differentiates from siblings by explicitly contrasting with image capture and hit-testing, and explains a unique benefit (naming elements a hit-test cannot reach). This leaves no ambiguity about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to prefer this tool: when a cheaper alternative to a screenshot is needed and when hit-testing cannot identify nested elements like inputs inside a Chromium page. It also notes the tool is read-only and safe while the user works. It does not explicitly state when not to use it or name alternative sibling tools, but the context is strong enough for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Type a text string into a window character-by-character (window-targeted PostMessage, not global keystrokes). MUTATING (soft): serialized across agent sessions; no physical input is injected so it can run while the user types — but the window may be RAISED to foreground, so avoid typing into a window the user is currently working in.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| text | No | ||
| title | No | ||
| delay_ms | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and excels: it discloses the mutation behavior (soft), serialization across agent sessions, lack of physical input injection, the potential to raise the window, and a practical user-safety warning. This is far beyond typical descriptions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no fluff. The main action and key constraints are front-loaded, and every phrase adds value—from the PostMessage detail to the mutating/serialization note to the user-warning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter tool with zero schema descriptions, the description is insufficient. It does not clarify parameter semantics, required vs optional fields, or how to target the window (hwnd vs title). Although an output schema exists, the description still leaves critical calling details unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for its 4 parameters, so the description must compensate. It implies 'text' is the string and that the window is targeted (via hwnd or title), but it does not explain what each parameter does, how hwnd and title interact, or what delay_ms controls. This leaves significant ambiguity for an agent constructing a call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool types a text string into a window character-by-character, specifically via window-targeted PostMessage rather than global keystrokes. This is a specific verb and resource, and it differentiates from sibling tools like tool_send_keys that likely use global keystrokes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context: window-targeted, can run while the user types, and warns against typing into a window the user is actively using. However, it does not explicitly name alternative tools like tool_send_keys, so the 'when-not' guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.3.1tool_capture_windowtool_click_attool_clipboard_readtool_clipboard_writetool_coexistence_statustool_cursor_positiontool_element_actiontool_element_action_attool_element_at_pointtool_find_elementstool_find_windowtool_get_window_recttool_list_displaystool_list_windowstool_open_applicationtool_read_elementtool_send_keystool_skyshottool_type_textScored across 19 tools
Most tools have clearly distinct purposes (list_windows vs find_window vs get_window_rect each serve different needs), and the description-heavy element discovery trio (element_at_point, skyshot, find_elements) is distinguished by use case. The main ambiguity is element_action vs element_action_at — the same action vocabulary addressed by different handles — and the three 'reach an element' paths could cause misselection, though the descriptions mitigate this.
Action tools follow a solid verb_noun pattern (list_windows, capture_window, send_keys, open_application), but several query/diagnostic tools break it by dropping the verb: cursor_position, skyshot, coexistence_status, and element_at_point are noun phrases or brand names. The uniform tool_ prefix and readable names keep it navigable, but the mixed convention is noticeable.
At 19 tools this sits in the heavy range, and the count is inflated by redundant addressing paths (element_action vs element_action_at, and three separate element-discovery tools). However, the server genuinely spans several sub-domains — window management, element access, input injection, clipboard, display enumeration, and coexistence policy — so the breadth is defensible.
The surface covers the full automation loop well: discover (find_window, find_elements, element_at_point, skyshot), read (read_element, capture_window, clipboard_read), and act (click_at, type_text, send_keys, element_action, element_action_at, open_application). Notable gaps are window manipulation (no close/resize/move/activate tool — only geometry reads) and no drag/gesture operation, but agents can work around these.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
Melaya is a remote MCP server. It gives an assistant hands on your own Android phone and browser: it reads the screen through the accessibility tree, then taps, types and navigates inside the apps and sites you allow-list, with no per-app API. It also builds, schedules and runs agent pipelines across 6k+ connected tools. OAuth 2.1, nothing to install.
Eyes and hands on real Windows PCs — observe, click, type via Glasswarp API.
MCP server for visual regression testing: triage a PR's UI diffs from your coding agent.
