Windows 电脑操控的 MCP 服务器 + agent 技能：无障碍元素动作优先，截图只是兜底； 带跨会话仲裁器——多个 agent 共享一台电脑时自动串行化，并在你正在使用电脑时主动让行。
本仓库只含 MCP 服务器与技能本身，对任何 stdio MCP 客户端保持中立 （dsh / Claude Code / Codex / Cursor / Cline / ZCode …）—— 不依赖 dsh 才能用。
平台语义（0.3.1 起）：工具只在 Windows 上可用——它们驱动 user32/kernel32 与 UI Automation。
但包本身在别的平台也能导入、服务器也能启动，并照常回应 tools/list，所以任何客户端
或目录抓取器都能枚举到这 19 个工具及其完整 schema；真去调用某个工具时得到的是一句明确的
「需要 Windows」错误，而不是进程根本起不来。0.3.0 在导入时就抛错，导致这类抓取完全看不到它
（tests/linux-handshake.py 是这条性质的回归测试，CI 在 ubuntu-latest 上跑它）。
一个 stdio MCP 服务器，暴露 19 个工具：
观测（只读，随时可调）：skyshot（把窗口读成带 diff 的紧凑文本树，比截图小三个数量级）、
element_at_point、read_element、find_elements、capture_window（DPI 感知 + 客户区裁剪）、
list_windows / find_window / get_window_rect、list_displays、cursor_position、
clipboard_read、coexistence_status
元素动作（软门：跨 agent 串行，无物理输入注入）：element_action /
element_action_at——press / set_value / select / toggle / expand / collapse /
scroll_into_view / focus，直接作用于 UIA 元素，不抢焦点、不关心 z 序
物理输入（硬门：跨 agent 串行 + 人机让行）：click_at（先元素路径后裸事件）、
send_keys、type_text（PostMessage 定向）、clipboard_write、open_application
每个动作返回回执而非自述成功：action_sent / effect_verified / foreground_changed /
user-active / arbiter-busy。「调用被接受」和「效果发生」是两件事——工具替 agent 分清。
Related MCP server: openowl
Windows 侧已经有好几个成熟的开源实现。dsh-cua 的差异集中在**「和人类共用一台机器」**这一点上：
dsh-cua | ||||
元素动作走 UIA 模式（不抢焦点、不关心 z 序） | ✅ | ✅（ax 档） | ❌ 只有坐标点击 | 经 cua-driver |
检测到人正在输入 → 拒绝 | ✅ | ❌ | ❌ | ❌ |
跨 agent 串行化（多进程） | ✅ named mutex | ❌ | ❌ | ❌ |
逐动作效果断言 | ✅ | 报告交付档位 | ❌ | ❌ 仅 |
抢前台副作用度量 | ✅ | ❌ | ❌ | ❌ |
工具数 | 19 | 59 | 15 | 6 |
关键区别是两件常被混为一谈的事：
「不抢焦点」是机制保证 —— 走 UIA 模式或定向 PostMessage，物理上不碰光标和键盘焦点。
cua-driver 有（ax 档）；ahk-mcp 其实没有 —— 它只有坐标点击，每次都移动真实光标。
「你一动就让路」是时间保证 —— 用 GetLastInputInfo 读人类最后一次输入的年龄，
检测到你正在用就等待，超时则拒绝（user-active）而不是硬上。上表另外三个实现里都没有这一条。
effect_verified 同样是同类没有的：它把「调用被接受」和「效果发生」分开，给出三态
（true 变化符合预期 / false 接受了但没变并降级为失败 / null 无可比状态即未确认）。
同类的替代做法是动作后重新观察一次，把判断留给模型。
dsh-cua 不做的事（先说清楚，避免误解）：没有像素/视觉接地——树表达不了的界面（canvas、 游戏、远程桌面）用不了；没有录制回放；没有隔离沙箱。这些各有更合适的方案。
需要 Windows x64 + 交互式桌面会话 + Python ≥3.10 才能真正操控桌面。
（包在 Linux/macOS 上同样可以安装与启动，tools/list 正常返回，只是调用工具时会明确
报「需要 Windows」——见上文"平台语义"。）
# 方式一：uvx 零安装（推荐）
uvx dsh-cua # 直接运行 stdio MCP server
# 方式二：pip
pip install dsh-cua
# 方式三：从源码
pip install git+https://github.com/Hutusion/dsh-cua.git三种方式装完后，用 python -m dsh_cua 起服务：
python -m dsh_cua # 不依赖 PATH 上的任何可执行文件为什么不写
dsh-cua-server：pip 会把 console script 装进解释器的Scripts目录， 而那个目录不一定在 PATH 上 —— 实测 stock python.org 3.12 的 User 与 Machine PATH 都不含它，于是pip install dsh-cua成功、dsh-cua-server却报 command not found。python -m不需要任何 PATH 条目。console script 仍然提供，PATH 里有它时可用。方式一/二现在都可用：包已发布在 PyPI（https://pypi.org/project/dsh-cua/）。 若哪一天
uvx/pip报 404，用方式三 —— 它总是可用。
任何 MCP 客户端，把 server 命名为 win32（skill 的工具名约定是 mcp__win32__*）。
python -m（不依赖 PATH，推荐）：
{ "mcpServers": { "win32": { "command": "python", "args": ["-m", "dsh_cua"] } } }uvx（PyPI 发布后）：
{ "mcpServers": { "win32": { "command": "uvx", "args": ["dsh-cua"] } } }更多形状见 examples/：Claude Code / 通用客户端 / dsh 的 cordis.patch.yml 片段 /
想让模型看懂截图时所需的路由模态声明（tr-route-settings.yml）。
skill/computer-use/SKILL.md 是配套的使用教条：观察→定位→动作→复核
的循环、回执语义、重试安全、与人类共存的纪律。没有它模型也能用工具，但有了它模型会自己选对
路径——实测差别很大。把它复制进你的技能目录即可：
# Claude Code / 通用 agents
cp -r skill/computer-use ~/.agents/skills/
# dsh
cp -r skill/computer-use ~/.dsh/skills/级别 | 覆盖操作 | 门 |
只读 | 观测类 12 个工具 | 不进门，随时可调 |
软门 | 元素动作、PostMessage 打字、剪贴板写、启动应用 | 跨 agent 互斥锁（named mutex，多进程自动串行） |
硬门 | 裸点击、全局热键 | 互斥锁 + |
诚实边界：让行是合作协议不是硬保证（注入前 150ms 紧检查已尽量收窄窗口）；
个别应用连 set_value 都会自激活（回执会如实报告 foreground_changed）；
同一窗口的「双人操作」没有技术解，别和 agent 同时操作同一个窗口。
python tests/verify-coexistence.py # 25 项：零输入证明 / 跨进程互斥 / 合成人机争用 / 杀开关
python tests/verify-p0-fixes.py # 0.2.0 修掉的三个 P0：每项在修复前必失败测试不需要真人配合——「用户输入」由一次真实 1px 光标移动合成，跑完还原。
测试需要真实交互式桌面会话（部分检查要创建窗口并用 UIA 寻址），所以不能在 GitHub 托管的
runner 上跑。CI 覆盖的是不需要桌面的那部分：打包安装、模块导入、diff 索引与树行转义的回归、
仲裁器的判定逻辑 —— 见 .github/workflows/ci.yml。
python tests/ci-desktop-free.py # 上面这些的本地等价物，不需要桌面MIT
Capture a window screenshot and SAVE IT TO A FILE. Provide hwnd or title substring. Crops to the window's CLIENT area by default, so image pixel (0,0) is the same point click_at calls (0,0) — no frame offset to guess. Returns bounds = the screen rect (x,y,width,height) the image maps to, plus scale (multiply image pixels by it to get screen pixels; 1.0 unless you asked for downscaling). Also reports dpi_verified; when false, image pixels and input coordinates cannot be safely paired. Defaults to JPEG q80 — a 1080p shot is ~6-10x smaller than PNG, which is what keeps a long session's request body under the provider's 32 MiB cap. The bytes are NOT inlined as base64 by default (that costs ~12,500 tokens per shot as text); read the saved_path with read_image instead, which attaches the same picture for ~50 tokens. Pass include_data_uri=true only when a caller genuinely cannot read a file. Read-only — no input injection, no focus change.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| title | No | ||
| max_dim | No | ||
| quality | No | ||
| save_path | No | ||
| client_only | No | ||
| image_format | No | jpeg | |
| include_data_uri | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure — and it goes far beyond basic mutation info. It discloses read-only status ('no input injection, no focus change'), explains the JPEG-vs-PNG size tradeoff relative to the 32 MiB provider cap, quantifies token costs (~12,500 vs ~50 tokens), and warns about the dpi_verified=false pairing hazard. This is exemplary behavioral transparency with zero annotation support.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every clause earns its place — the coordinate alignment, file/scale/bounds return contract, size rationale, and token economics all carry operational value. It's front-loaded with the primary purpose and cropped-client-area detail. Slightly essay-like ('which is what keeps a long session's request body under the provider's 32 MiB cap') but purposeful; each sentence advances understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 8-parameter tool with no annotations and zero schema coverage, this description is remarkably complete: it explains the return contract (bounds, scale, dpi_verified) even though an output schema exists, covers file-saving semantics, and anticipates the exact failure mode an agent would hit (huge base64 payloads). Nothing critical an agent needs to invoke it correctly or interpret its output is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate — and it does, covering nearly every parameter in prose: hwnd/title (provide either), client_only (client area cropping), max_dim (downscaling), quality (JPEG q80 default), include_data_uri (token cost and base64 inlining), and save_path (read with read_image). A minor gap is that image_format is never explicitly named, and some parameters aren't labeled by their schema property names, but the intent is conveyed well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Capture a window screenshot and SAVE IT TO A FILE') and adds a distinguishing detail — cropping to the client area with aligned coordinate systems — that differentiates it from siblings like tool_skyshot. The purpose is unambiguous and the resource (window) is clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit input guidance ('Provide hwnd or title substring') and a conditional rule for when to pass include_data_uri ('only when a caller genuinely cannot read a file'), plus a recommended alternative (read_image). However, it never names sibling tools as explicit exclusions (e.g., when to prefer tool_skyshot or tool_get_window_rect over this), so it lacks direct when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Click at client-area coordinates within a window. Coordinates relative to window content area (0,0 = top-left). Tries the ELEMENT path first (resolve the point via UI Automation and invoke that element) because it needs no focus and does not care about z-order; falls back to a raw cursor click, which does. The result's method says which was used: 'ax_press' or 'raw_event'. MUTATING: serialized across agent sessions; the raw_event path ALSO YIELDS TO THE USER (waits for input-quiet, then refuses with user-active rather than fighting them for the cursor); the ax_press path injects no physical input. The raw path verifies the addressed window actually owns that screen point and returns success=false instead of clicking another window. Pass dry_run=true to check where the click would land without moving the cursor, clicking, or taking any gate.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| hwnd | No | ||
| title | No | ||
| prefer | No | element | |
| dry_run | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It discloses mutation, serialization, user yielding for the raw path, z-order/focus dependencies, window ownership verification, and dry_run side-effect-free behavior. This is unusually rich and goes well beyond a simple 'click'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but front-loaded with the core purpose and every sentence adds a distinct behavioral fact. The MUTATING and dry_run sentences are dense but necessary; could be slightly tightened but overall well structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-param tool with no annotations and no schema descriptions, the description covers most behavioral context but leaves title and full prefer semantics undocumented. It is close to complete but not fully sufficient for all invocation decisions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description must document parameters. It explains x/y are client-area coordinates, dry_run prevents effects, prefer relates to element vs raw, and hwnd as 'addressed window'. Title is not documented, and prefer values beyond default 'element' are unstated, so compensation is strong but incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action ('Click at client-area coordinates within a window') and gives coordinate meaning. It explains internal click paths but does not explicitly differentiate from sibling tools like tool_element_action_at or tool_send_keys, so it lacks direct sibling contrast.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how the tool behaves (element first, raw fallback, dry_run) and when it is safe to use, but it never tells the agent which sibling to choose instead. No exclusions or comparisons to alternatives are given, so usage guidance is implied by the action itself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read the current clipboard text. Read-only. Non-text clipboard content reports empty text rather than failing.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses a key behavioral trait: non-text clipboard content reports empty text rather than failing. This goes beyond the basic read-only annotation and helps the agent anticipate edge cases. However, it does not mention potential side effects (e.g., clipboard access permissions) or whether the clipboard is cleared after reading, but for a read operation this is minor.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no wasted words. It front-loads the core action ('Read the current clipboard text'), then adds the read-only note and the edge-case behavior. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter read tool, the description is nearly complete. It states what it does, the read-only nature, and the behavior for non-text content. The only minor gap is not describing the exact return format (e.g., whether it returns a string or an object), but the output schema exists and likely covers that. The description is sufficient for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description does not need to explain parameter semantics. The schema is empty and coverage is 100%, meaning there is nothing undocumented. The description adds value by clarifying the return behavior for non-text content, which is more useful than parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reads the current clipboard text, using a specific verb ('Read') and resource ('current clipboard text'). It also distinguishes itself from the sibling tool_clipboard_write by focusing on reading rather than writing. The behavior for non-text content is explicitly described, which further clarifies its purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool: whenever an agent needs to retrieve the current clipboard text. It does not explicitly name alternatives or exclusions, but the sibling list includes tool_clipboard_write, and the read-only nature is clear. The context is sufficient for an agent to select this tool over writing to the clipboard.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Replace the clipboard contents with text. MUTATING (soft): serialized across agent sessions — and it DESTROYS whatever the user last copied, so do not use it casually; announce it when you do.
| Name | Required | Description | Default |
|---|---|---|---|
| text | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the behavioral burden. It explicitly labels the tool as MUTATING, warns that it DESTROYS the user's last copied content, mentions serialization across sessions, and instructs the agent to announce the action. This exceeds typical disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no fluff. The primary action is front-loaded, and the critical warnings are placed immediately after, making the most important information easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter write tool, the description covers the action, destructive consequences, persistence behavior, and usage caution. The presence of an output schema means return-value details are unnecessary here, so the description is complete for safe invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no descriptions for the single 'text' parameter, and the description only vaguely maps to it via 'with text.' This provides basic meaning but does not fully compensate for the 0% schema coverage, so it lands at an adequate but not exceptional level.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Replace the clipboard contents with text,' which is a specific verb and resource. It clearly identifies the tool as the write counterpart to clipboard_read and leaves no ambiguity about its function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises against casual use and instructs the agent to announce the action, giving clear context for safe usage. It does not explicitly name an alternative tool or state when not to use it beyond the strong caution, so it falls just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Report the coexistence policy in force: whether the admission gate is enabled, the quiet/max-wait/mutex-timeout tunables, the pre-injection final-check window, named-mutex health, and how long ago the last physical input happened. Read-only — call it when an action was refused with user-active or arbiter-busy and you want the numbers behind the refusal.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does disclose the key behavioral trait: 'Read-only.' It also sets expectations about the returned data by listing tunables, health, and timing information. It does not cover access prerequisites or failure behavior, but those are less critical for a stateless status report.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence with the core purpose front-loaded before a colon-delimited list of report contents. Every phrase earns its place, and the description avoids padding while remaining technically precise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only status tool with an output schema, the description provides both when to invoke it and what data to expect. The output schema covers the return structure, so nothing an agent needs to call this tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so there is no schema description to compensate for; the 0-param baseline is 4. The description still adds semantic context by enumerating what the caller receives, which is useful even though there are no inputs to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Report') and resource ('coexistence policy in force'), then enumerates exact contents such as the admission gate, tunables, named-mutex health, and last physical input. This clearly distinguishes it from sibling action tools like tool_click_at and tool_send_keys, which perform UI operations rather than report policy state.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to call it: 'call it when an action was refused with user-active or arbiter-busy and you want the numbers behind the refusal.' It also frames the tool as read-only, making it clear this is a diagnostic fallback rather than a routine action; no alternative tool is needed because no sibling offers this state.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Current cursor position plus the screen size, both physical pixels. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states 'Read-only', indicating no side effects, and specifies that values are in physical pixels, which is critical for coordinate interpretation. This provides meaningful transparency beyond a bare 'returns position' statement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that conveys the primary purpose (cursor position and screen size) and a key qualifier (physical pixels), followed by the 'Read-only' trait. There is no wasted prose, and the most important information appears first.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (no parameters) and an output schema exists (not shown but indicated), so the description need not explain return types. It covers purpose, units, and side-effect nature. Minor details like multi-monitor behavior are not addressed, but for a basic read tool this is adequate. The presence of an output schema reduces the burden further.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4 per the rubric. The description adds no parameter semantics (there are none to explain), but it does clarify the output semantics (position and screen size), which is useful context. No gaps exist since there are no parameters to document.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns the current cursor position and screen size in physical pixels, which is a specific purpose distinct from sibling tools like tool_click_at (which manipulates coordinates) or tool_get_window_rect (which returns window bounds). The 'Read-only' phrase further disambiguates it as a query operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage when an agent needs the cursor position or screen dimensions, but it does not explicitly mention when to choose this tool over alternatives or list exclusions. Since the purpose is self-contained and no alternative is cited, the guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Perform an action on a UI element by the ref from element_at_point — this is the focus-free path: a UI Automation pattern is delivered to the element, so the window is never raised and z-order never matters. MUTATING (soft): serialized across agent sessions; NO physical input is injected, so it can run while the user types — just do not operate the very window the user is working in. action must be one of the actions listed for that element (press, set_value, select, toggle, expand, collapse, scroll_into_view, focus); set_value also needs text. Refuses with the supported list when the element does not offer the action.
| Name | Required | Description | Default |
|---|---|---|---|
| ref | Yes | ||
| text | No | ||
| action | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral burden and does so thoroughly. It discloses mutation ('MUTATING (soft)'), serialization across sessions, lack of physical input injection, the ability to run during user typing, and refusal behavior with the supported action list. This is far beyond the minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place. It front-loads the core purpose, then adds constraints, safety context, and parameter semantics in a logical order. No filler or tautology.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters and an output schema, the description is complete. It covers action constraints, the source of ref, the mutating behavior, safety implications, and refusal behavior. Return values are not explained, but the output schema already handles that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains ref as coming from element_at_point, enumerates valid action values, and explicitly ties text to set_value. This adds substantial meaning to all three parameters, including the optional text parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Perform an action'), a precise resource ('a UI element by the ref from element_at_point'), and a distinguishing property ('focus-free path'). The description clearly differentiates this from coordinate-based or physical-input tools like tool_click_at and tool_element_action_at.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context: use this path when you have a ref from element_at_point and want a focus-free action. It also gives a concrete usage caveat ('do not operate the very window the user is working in') and notes it can run while the user types. It does not explicitly name alternatives or exclusions, but the guidance is strong enough to infer appropriate use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Act on the element that line index of the last skyshot named. This is the focus-free path for elements element_at_point cannot reach: the tree names the element, so the action goes to it and the window is never raised. MUTATING (soft): serialized across agent sessions; no physical input injected. action is one of press, set_value, select, toggle, expand, collapse, scroll_into_view, focus; set_value also needs text. Take a fresh skyshot if the UI has changed — a stale index is refused rather than acted on.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | Yes | ||
| text | No | ||
| index | Yes | ||
| action | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool is 'MUTATING (soft)', is serialized across agent sessions, injects no physical input, never raises the window, and refuses stale indexes rather than acting on them. These are all meaningful behavioral traits that an agent needs to know.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences pack in the core action, the differentiator, behavioral caveats, parameter options, and freshness guidance. Nothing is redundant, and the most important information (what the tool does and when to use it) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with four parameters and an output schema, the description covers the essential decision points: what index means, what actions are available, the soft-mutation semantics, and the stale-index safeguard. It doesn't detail every possible error or the return format, but the output schema presumably handles that. The description is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must explain the parameters. It does: it enumerates the valid action values, notes that set_value additionally requires text, and explains that index refers to a line in the last skyshot. It does not elaborate on hwnd or the exact meaning of each action, but the critical disambiguation is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Act on') and resource ('element that line index of the last skyshot'), and immediately distinguishes itself from a sibling tool by calling itself the 'focus-free path' for elements element_at_point cannot reach. The list of possible actions further clarifies its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names the sibling alternative (element_at_point) and gives a concrete condition for when to use this tool instead: 'for elements element_at_point cannot reach'. It also advises taking a fresh skyshot when the UI changes, which is a clear operational rule. No other alternatives are mentioned, but the primary decision point is covered.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Identify the UI element at a PHYSICAL screen point via Windows UI Automation. Returns its name, role, rect, and which actions it supports, plus an opaque ref for read_element. This is how you turn 'what I see in the screenshot' into something addressable without clicking: coordinates here are the same space as capture bounds (both physical pixels). Read-only — moves no cursor and changes no focus.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explicitly states 'Read-only — moves no cursor and changes no focus', which is a key behavioral disclosure. It also mentions the return of an opaque ref. It doesn't cover error conditions or edge cases like no element at the point, but for a read-only query the main behavioral trait is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, with no filler. The core purpose is front-loaded, followed by return values and the crucial coordinate-space clarification, and ends with a safety note. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has only two parameters and an output schema, so the description doesn't need to explain return values. It covers the use case, coordinate space, and read-only behavior. The only minor gap is not addressing the case where no element exists at the point, but the output schema likely handles that. Overall it is complete enough for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains that x and y are physical pixel coordinates in the same space as capture bounds, which is critical for correct usage. It also emphasizes 'PHYSICAL screen point' to avoid coordinate-space ambiguity. This adds meaning well beyond the bare integer type in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb ('Identify') and resource ('UI element at a PHYSICAL screen point via Windows UI Automation'), then lists exactly what is returned (name, role, rect, supported actions, opaque ref). It also distinguishes itself from siblings by framing it as the way to turn a screenshot point into an addressable element without clicking, clearly separating it from click_at and read_element.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states when to use it: when you have a physical screen coordinate from a screenshot and want to address the element without clicking. It notes the coordinate space matches capture bounds. It does not explicitly name alternatives or state when not to use it, but the purpose is clear enough that an agent can infer it is the right tool for point-based element lookup.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Find UI elements by role and/or name substring and return refs that can be acted on with element_action. This is how you reach an element that cannot be hit-tested — the point of naming it is that you no longer need a pixel. role is the lowercase role from skyshot (button, edit, document, list_item, ...). At least one filter is required: an unfiltered call would be a tree dump, so use skyshot for that. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| role | No | ||
| title | No | ||
| max_results | No | ||
| automation_id | No | ||
| name_contains | No | ||
| include_offscreen | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the behavioral burden. It discloses read-only behavior, the need for at least one filter, and the consequence of an unfiltered call (tree dump). It does not mention edge cases like no matches or offscreen behavior, but the core safety and boundary characteristics are clearly stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: it front-loads the primary action and output, then gives usage rationale, role semantics, filter requirement, and safety note. Every sentence contributes meaningful guidance with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 7-parameter tool with no annotations, the description covers the core intent, the key role semantics, the mandatory filter condition, and the read-only nature. An output schema exists, so return-value details are not required. The main omission is fuller parameter-level guidance, but the description is sufficient for competent selection and basic correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the 'role' parameter (lowercase role from skyshot, with examples) and implies 'name_contains' means substring matching. However, it leaves hwnd, title, automation_id, max_results, and include_offscreen essentially unexplained, so the compensation is only partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource ('Find UI elements'), states the matching criteria ('by role and/or name substring'), and explains the output ('return refs that can be acted on with element_action'). It also distinguishes itself from pixel/hit-test based tools, so an agent can separate it from siblings like element_at_point and skyshot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use this tool: to reach an element that cannot be hit-tested, because naming it removes the need for a pixel. It also gives an exclusion: an unfiltered call would be a tree dump, so use skyshot for that. The 'at least one filter is required' rule further guides correct invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Find a window by title substring and return its hwnd and geometry. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| title | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the tool is read-only, which is a key behavioral trait, and states the return content (hwnd and geometry). However, it does not mention behavior when no match is found, case sensitivity, or whether multiple matches are possible. This is minimal but not absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that states purpose and read-only nature efficiently. It could include a bit more detail without losing conciseness, but it is appropriately brief and wastes no words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists, return values are covered. The tool is simple with one parameter. Missing details include behavior on no match, multiple matches, and case sensitivity, which could affect an agent's interpretation. The description is adequate but not fully complete for all potential edge cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain the parameter. It states the 'title' is used for substring matching, which adds meaning beyond the schema's bare type and default. It does not clarify edge cases like empty string behavior, but the core semantic is conveyed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Find'), a specific resource ('window'), and the distinguishing mechanism ('by title substring'), and specifies the returned data ('hwnd and geometry'). This clearly differentiates it from sibling tools like tool_list_windows and tool_get_window_rect, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a clear use case: given a title substring, find a window. However, it does not explicitly mention alternatives or when not to use this tool (e.g., when listing all windows or retrieving geometry of an already known window). The usage guidance is adequate but not explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Get detailed position and size info for a window: absolute position, dimensions, and client area size. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| title | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states 'Read-only,' which is a key safety trait, but it does not disclose other behaviors: what happens if the window is not found, whether hwnd or title is used preferentially, whether the window must be visible, or any error conditions. For a tool with two optional parameters, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the core purpose and includes the read-only safety note. Every word earns its place; there is no fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the output schema exists to explain return values, the description does not cover essential context: how to choose between hwnd and title, what happens if neither is provided (since both have defaults), or whether the tool requires a valid window handle. For a tool with two optional parameters, this is incomplete and could lead to incorrect invocations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description does not explain the parameters at all. It does not clarify that hwnd and title are alternative identifiers, which one takes precedence, or how they interact. The description's mention of 'absolute position, dimensions, and client area size' describes the output, not the input. This leaves the agent guessing how to specify the target window.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: getting detailed position and size info for a window, including absolute position, dimensions, and client area size. This is a specific verb (get) and resource (window), and it distinguishes from siblings like tool_find_window (which likely finds handles) and tool_capture_window (which captures images). It is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites (e.g., need a window handle first) or scenarios where this should be preferred over tool_list_windows or tool_find_window. The description is purely declarative without any contextual direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List monitors: index (1-based), primary flag, device name, and the monitor plus work-area rectangles in physical pixels. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full behavioral burden. It explicitly says 'Read-only', which communicates non-mutating safety, and specifies the exact information returned, including the physical-pixel coordinate context. This is adequate for a simple list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense sentence front-loads the action and resource, then lists the output fields in a readable sequence. Every phrase earns its place, and the read-only note is a useful signal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool takes no parameters, has an output schema, and the description adds semantic meaning about the returned items (work-area rectangles, physical pixels, primary flag). An agent has everything it needs to select and invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description does not need to explain parameter semantics. The base baseline of 4 applies; there is nothing missing and the description adds no irrelevant parameter information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List monitors') and resource ('monitors') and enumerates the exact fields returned (index, primary flag, device name, monitor and work-area rectangles in physical pixels). It clearly distinguishes from display-related siblings by scope and output details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance about when to use this tool versus alternatives like tool_get_window_rect or tool_cursor_position. The description implies a read-only enumeration use case but does not state exclusions or conditions that would route an agent to this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List all visible top-level windows with title, hwnd, PID, position and size, sorted largest first. Read-only — safe while the user works.
| Name | Required | Description | Default |
|---|---|---|---|
| filter_title | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly states the tool is read-only and safe while the user works, and it discloses the sorting behavior and scope ('visible top-level windows'). This goes beyond the schema and gives an agent confidence about side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the action and resource, then packs in the key output fields and ordering. Every clause earns its place with no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The output schema exists, so return values are presumably covered elsewhere. The description covers purpose, scope, sorting, and safety, but it omits filter_title semantics and does not provide guidance on when to prefer this over sibling tools. For a simple list tool this is acceptable but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The only parameter, filter_title, is not mentioned in the description. Since schema description coverage is 0%, the description should explain what filtering means (e.g., substring match, case sensitivity), but it does not. The parameter name and default empty string imply optionality, but no semantic detail is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('List'), a precise resource ('all visible top-level windows'), and the exact fields returned (title, hwnd, PID, position, size). It also mentions sorting behavior, which distinguishes it from sibling tools like tool_find_window or tool_get_window_rect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool: when you need an overview of visible top-level windows. It also notes the operation is read-only and safe while the user works, which is useful context. However, it does not explicitly contrast this tool with alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Open a file, folder, or URI with the shell's default handler — like double-clicking it in Explorer: an .exe path, a document, or a URL. MUTATING (soft): serialized across agent sessions; the launched app may take focus.
| Name | Required | Description | Default |
|---|---|---|---|
| target | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it discloses important behavioral traits: it is labeled MUTATING (soft), serialized across agent sessions, and may take focus. This goes beyond a bare 'open' statement. The term 'soft' is somewhat informal, but the sentence adds genuinely useful side-effect context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with no filler: the first establishes the operation and examples, the second adds critical behavioral caveats. Every clause earns its place, and the key purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one required parameter) and has an output schema, so the description does not need to explain return values. It covers what is opened, how, and the relevant side effects (session serialization, focus stealing), making the definition complete for an agent deciding whether and how to call it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate for the single 'target' parameter. It does so by defining target as a file, folder, or URI and giving examples (.exe path, document, URL). It could add path-format or URI-scheme details, but the required meaning is clear enough for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Open') and a precise resource class ('a file, folder, or URI') via the shell's default handler, with concrete examples like .exe paths, documents, and URLs. This clearly distinguishes it from the UI-automation siblings, none of which launch external default-handler targets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear mental model — same as double-clicking in Explorer — so an agent knows to use it when it wants OS-level default handling rather than window/element manipulation. It does not explicitly name alternatives or exclusion conditions, but the sibling tools are sufficiently distinct that no close alternative needs to be ruled out.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Re-read a UI element by the ref returned from element_at_point. Use this to check whether an element is still there before acting on it; a ref goes stale when the UI is rebuilt, and that is reported rather than guessed. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| ref | Yes |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden. It discloses that the tool is read-only and explains that stale refs are reported rather than guessed. It doesn't detail error handling or output format, but these are likely covered by the output schema. The key behavioral trait of staleness reporting is clearly communicated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the core action and source stated first, followed by the usage scenario and staleness behavior. Every word earns its place; there is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read tool with an output schema (even though not shown), the description covers purpose, usage, and the key behavioral nuance of staleness. The agent has enough to decide when to call it and what to expect, and the output schema presumably covers return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains that 'ref' is a reference returned by element_at_point and clarifies that it can become stale. This adds meaning beyond the bare 'string' type, giving the agent enough context to use the parameter correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('re-read') and resource ('UI element'), and clarifies the ref comes from element_at_point. It distinguishes itself by focusing on re-reading a specific element rather than performing actions on it, making its purpose clear and unique among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit guidance on when to use it ('Use this to check whether an element is still there before acting on it') and explains the stale ref scenario. It doesn't name alternative tools explicitly, but the context implies its role in the verification step. A slight gap is the lack of explicit 'when not to use' guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Send keyboard shortcut to a window. Example: keys=['ctrl','s']. For Alt+key menu navigation, include 'alt' as first key (or set use_alt_key). MUTATING: serialized across agent sessions via the admission gate, and global hotkeys additionally YIELD TO THE USER — recent mouse/keyboard activity makes them wait, then refuse with reason user-active instead of interrupting; switch to element actions or retry later. Also refuses when the window is not the foreground window, because injected keys would go to the wrong window.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| keys | No | ||
| title | No | ||
| use_alt_key | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are absent, so the description carries the full burden and it delivers: it discloses that the tool is mutating, serialized via admission gate, that global hotkeys yield to the user and refuse with reason user-active, and that it refuses when the target window is not the foreground window. That is rich behavioral disclosure beyond a bare 'send keys' statement. A 5 would require even more, like what exactly happens on failure, but this is substantial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence earns its place: the example, the Alt-navigation tip, the serialization/yield-to-user behavior, and the foreground-window precondition. No filler and information-dense.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a 4-parameter tool with no annotations but a present output schema and 18 siblings, the description is complete enough to invoke correctly: window targeting (hwnd/title), shortcut syntax, Alt-key mode, and two refusal conditions are all covered. The output schema exists to explain return values, so that is not a gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, and it does for the key parameters: keys get the example ['ctrl','s'], use_alt_key gets the 'alt as first key' guidance, and title/hwnd are the window-targeting pair. It doesn't explain the hwnd=0 default or the title fallback relationship explicitly, but it covers the two non-obvious parameters. Baseline would be 3 at 100% coverage; with 0% coverage and this much guidance, a 4 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: send keyboard shortcut to a window. Not fully differentiated from sibling tool_type_text, but it names the target and the key-list mechanism clearly, and the title is null so the description carries the load.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives when-to-use guidance: use for shortcuts like ctrl+s, use 'alt' first key for menu navigation. It also gives a when-not-to-use and an alternative: switch to element actions, or retry later, when user is active. This clearly routes the agent compared with siblings like tool_click_at, tool_type_text, and tool_element_action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read a window's UI as a compact TEXT tree instead of a screenshot — far cheaper than an image, and it names elements a hit-test cannot reach (e.g. an input inside a Chromium page, which element_at_point reports only as the enclosing 'document'). Each line is '{index} {indent}{role} {name}{Value: ...}{(state)}'. Every shot after the first is a DIFF against the previous one: unchanged lines are omitted, '' marks a changed line, '+' an added one, and removed indices are summarised as ranges. So a second call is usually a few lines, not the whole tree — an unchanged window costs about 100 characters. INDEXES ARE POSITIONAL: each shot numbers the tree it just walked, so a number is only valid against the shot that printed it. '' and '+' lines carry THIS shot's index and may be acted on; unchanged lines carry no index, so reaching one needs disable_diff=true for a full render. An index read from an older shot is not a durable handle — re-check it with a fresh shot before acting if the window may have changed. The result includes shot_key (the window the diff baseline belongs to — the baseline lives server-side and outlives your session, so this is how you confirm it was taken against the window you meant). Pass include_offscreen=true to include elements that are scrolled out or hidden. NOTE: this reads element VALUES, so the text can contain whatever is on screen in that window. Read-only — no input injection, no focus change; safe to call while the user works.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| title | No | ||
| max_depth | No | ||
| max_nodes | No | ||
| disable_diff | No | ||
| include_values | No | ||
| include_offscreen | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the behavioral burden, and it does so exceptionally. It explains the diff behavior between shots, the exact line format, positional index caveats, the server-side shot_key baseline, offscreen inclusion, and that values may contain arbitrary on-screen text. It also explicitly declares the tool is read-only with no input injection or focus change, which is critical behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but densely packed with necessary operational details. The core purpose is front-loaded, followed by output format, diff semantics, index validity, and safety. A few details could be tightened, but the length is justified by the tool's complexity and the absence of annotation support.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with no annotations, the description covers nearly everything an agent needs: output format, diff semantics, positional index pitfalls, shot_key behavior, offscreen handling, and read-only safety. It is slightly incomplete on how to target the window via hwnd or title and does not clarify max_depth or max_nodes, but the overall picture is very strong.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does explain disable_diff, include_offscreen, and indirectly include_values through the note about reading element values. However, it does not explain hwnd, title, max_depth, or max_nodes, although their names and defaults are somewhat self-evident. This is partial compensation, not full, so a 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action and resource: 'Read a window's UI as a compact TEXT tree instead of a screenshot'. It differentiates from siblings by explicitly contrasting with image capture and hit-testing, and explains a unique benefit (naming elements a hit-test cannot reach). This leaves no ambiguity about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to prefer this tool: when a cheaper alternative to a screenshot is needed and when hit-testing cannot identify nested elements like inputs inside a Chromium page. It also notes the tool is read-only and safe while the user works. It does not explicitly state when not to use it or name alternative sibling tools, but the context is strong enough for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Type a text string into a window character-by-character (window-targeted PostMessage, not global keystrokes). MUTATING (soft): serialized across agent sessions; no physical input is injected so it can run while the user types — but the window may be RAISED to foreground, so avoid typing into a window the user is currently working in.
| Name | Required | Description | Default |
|---|---|---|---|
| hwnd | No | ||
| text | No | ||
| title | No | ||
| delay_ms | No |
| Name | Required | Description |
|---|---|---|
No output parameters | ||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and excels: it discloses the mutation behavior (soft), serialization across agent sessions, lack of physical input injection, the potential to raise the window, and a practical user-safety warning. This is far beyond typical descriptions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no fluff. The main action and key constraints are front-loaded, and every phrase adds value—from the PostMessage detail to the mutating/serialization note to the user-warning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter tool with zero schema descriptions, the description is insufficient. It does not clarify parameter semantics, required vs optional fields, or how to target the window (hwnd vs title). Although an output schema exists, the description still leaves critical calling details unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage for its 4 parameters, so the description must compensate. It implies 'text' is the string and that the window is targeted (via hwnd or title), but it does not explain what each parameter does, how hwnd and title interact, or what delay_ms controls. This leaves significant ambiguity for an agent constructing a call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool types a text string into a window character-by-character, specifically via window-targeted PostMessage rather than global keystrokes. This is a specific verb and resource, and it differentiates from sibling tools like tool_send_keys that likely use global keystrokes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context: window-targeted, can run while the user types, and warns against typing into a window the user is actively using. However, it does not explicitly name alternative tools like tool_send_keys, so the 'when-not' guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
v0.3.1tool_capture_windowtool_click_attool_clipboard_readtool_clipboard_writetool_coexistence_statustool_cursor_positiontool_element_actiontool_element_action_attool_element_at_pointtool_find_elementstool_find_windowtool_get_window_recttool_list_displaystool_list_windowstool_open_applicationtool_read_elementtool_send_keystool_skyshottool_type_textScored across 19 tools
Most tools have clearly distinct purposes (list_windows vs find_window vs get_window_rect each serve different needs), and the description-heavy element discovery trio (element_at_point, skyshot, find_elements) is distinguished by use case. The main ambiguity is element_action vs element_action_at — the same action vocabulary addressed by different handles — and the three 'reach an element' paths could cause misselection, though the descriptions mitigate this.
Action tools follow a solid verb_noun pattern (list_windows, capture_window, send_keys, open_application), but several query/diagnostic tools break it by dropping the verb: cursor_position, skyshot, coexistence_status, and element_at_point are noun phrases or brand names. The uniform tool_ prefix and readable names keep it navigable, but the mixed convention is noticeable.
At 19 tools this sits in the heavy range, and the count is inflated by redundant addressing paths (element_action vs element_action_at, and three separate element-discovery tools). However, the server genuinely spans several sub-domains — window management, element access, input injection, clipboard, display enumeration, and coexistence policy — so the breadth is defensible.
The surface covers the full automation loop well: discover (find_window, find_elements, element_at_point, skyshot), read (read_element, capture_window, clipboard_read), and act (click_at, type_text, send_keys, element_action, element_action_at, open_application). Notable gaps are window manipulation (no close/resize/move/activate tool — only geometry reads) and no drag/gesture operation, but agents can work around these.
MCP server for progressive tool usage at any scale (see https://klavis.ai)
Melaya is a remote MCP server. It gives an assistant hands on your own Android phone and browser: it reads the screen through the accessibility tree, then taps, types and navigates inside the apps and sites you allow-list, with no per-app API. It also builds, schedules and runs agent pipelines across 6k+ connected tools. OAuth 2.1, nothing to install.
Eyes and hands on real Windows PCs — observe, click, type via Glasswarp API.
MCP server for visual regression testing: triage a PR's UI diffs from your coding agent.
