nested-display-mcp
Drives Wine and Windows 9x applications and installers inside a nested Wayland/Xwayland display with exact screenshots and reliable keyboard/pointer control, e.g. launching wine setup.exe and interacting with its windows via absolute clicks, chords and typed text.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@nested-display-mcpopen a 1024x768 display, launch the installer, and screenshot it"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
nested-display-mcp
An MCP server that runs an application inside a visible nested Wayland display and gives an agent exact screenshots and reliable keyboard/pointer control, without touching the host desktop's focus, cursor acceleration or window management.
Built for driving GUI apps, installers, emulators and old games (Wine/Windows 9x) where injecting input into the real desktop is fragile. Each task gets a real compositor, so screenshots are exact and input never depends on host focus.
┌─────────────┐ stdio ┌────────────────────┐ spawn ┌──────────────────────────────┐
│ MCP client │ ───────► │ nested-display-mcp │ ──────► │ sway (nested, one per display)│
└─────────────┘ └────────────────────┘ │ ├─ WAYLAND_DISPLAY=wayland-N │
▲ │ │ └─ Xwayland :M (Wine/X11) │
│ PNG + JSON │ └──────────────────────────────┘
└────────────────────────┴── grim / wlrctl / wtype / xdotoolWhy sway, and why this shape
Need | How |
A window the user can watch and move | nested sway with |
Exact, palette-correct screenshots |
|
Absolute pointer, no host focus |
|
Keyboard without focus games |
|
Wine / X11 apps | sway starts its own Xwayland; each display gets its own |
Knowing what is on screen |
|
Cage was tested and rejected: it refuses external Xwayland (xwayland_shell_v1 permission
denied), so it cannot run Wine. Gamescope works but has no screencopy (no grim) and only one
window; it stays a possible future "game mode" backend.
Related MCP server: deskwright
Requirements
System packages (Fedora names):
sudo dnf install -y sway grim wlrctl wtype xdotoolOn Debian/Ubuntu: sudo apt install -y sway grim wlrctl wtype xdotool; on Arch:
sudo pacman -S sway grim wlrctl wtype xdotool. Node.js 20 or newer is also required.
The host must have a Wayland session (sway nests inside it) or an X11 session with XWayland;
the default x11 backend needs a host DISPLAY. The MCP server inherits WAYLAND_DISPLAY,
DISPLAY and XDG_RUNTIME_DIR from the client that starts it.
Build
npm ci # or npm install
npm run buildInstall
The repo is self-contained: the root is both the agent skill (SKILL.md) and the MCP server
(dist/index.js), so a single checkout gives you both.
git clone https://github.com/gashcrumb/nested-display-mcp.git ~/.agents/skills/nested-display
cd ~/.agents/skills/nested-display
npm ci && npm run buildThen register the server with your client (below) and restart it. Some clients look in their own
skills directory instead of ~/.agents/skills; see
docs/installation.md for client-specific setup, updating and uninstalling.
Smoke-test without an agent:
npm run smoke # opens a real nested display, clicks/types, then closes itRegister with your MCP client
The server is a stdio MCP server: run node /path/to/nested-display-mcp/dist/index.js and speak
MCP on stdin/stdout. For opencode, that looks like this in ~/.config/opencode/opencode.jsonc:
"mcp": {
"servers": {
"nested-display": {
"type": "local",
"command": [
"/usr/bin/node",
"/home/<you>/.agents/skills/nested-display/dist/index.js"
],
"disabled": false
}
}
}Use command -v node for the right binary (nvm paths differ per machine). Other MCP clients use
the same command/args pair in their own config format. Restart the client so the tools are
discovered.
Tools
Tool | Purpose |
| Start a display ( |
| Inspect or tear down displays ( |
| Full-frame PNG, optional region ( |
| Crop to one window's rect |
|
|
| Window and display management |
| Absolute output coordinates (or |
| Delta motion - for PS/2 guests, captured pointers and camera controls |
|
|
| Press, move in steps, release (X11 targets only) |
|
|
| Text entry and chords ( |
| Wait on a regex over title/app_id/class |
| Sleep (e.g. after launching an app) |
All tools take an optional id; with a single display open it can be omitted.
Example
display_open { title: "setup", width: 1024, height: 768, app: "wine", args: ["setup.exe"], env: { WINEDLLOVERRIDES: "ddraw=n" } }
wait_for_window { pattern: "Setup" }
screenshot {}
click { x: 512, y: 384 }
type { text: "hello" }
key { combo: "alt+Return" }
display_close {}Input routing (important)
A target whose
shellisxwaylandgets XTest keyboard input: focus first, then type globally.xdotool type --windowdrops characters in some apps (observed with Wine Notepad).A Wayland target gets wtype / wlrctl. Do not mix the two paths in the same session: a virtual-keyboard keymap upload changes Xwayland's keymap and XTest text then arrives garbled. Never use
wtypeon an X11 target - a stray mapping can arrive as Escape and close the app.Pointer moves always go through the compositor:
wlrctl pointer move -100000 -100000(clamp to the origin), then move by the exact delta.wlrctlmotion is unaccelerated, so this is absolute and exact. XTest warps are not used for positioning because polling toolkits (raylib/GLFW, SDL) never see a motion event from them, leaving their cached cursor position stale. For X11 targets the position is verified withxdotool getmouselocationand a warp is used only as a fallback.The first pointer event after a display is created is swallowed while Xwayland takes pointer focus; the server sends one throwaway move per display before the first real interaction.
Clicks hold the button for 100 ms by default (
hold_ms). Polling UIs checkIsMouseButtonPressedonce per frame, so an atomic press+release between frames is lost.Key chords hold for 80 ms by default (
hold_msonkey) for the same reason: a tap that lands between two polls is invisible toIsKeyPressed.dragis X11-only:zwlr_virtual_pointer_v1has no button-hold action.
Testing
npm run build
npm run smoke # X11 client: open, wait, screenshot, type, key, click, resize, close
node scripts/wayland-test.mjs # Wayland client (foot): wtype + virtual pointer
node scripts/multi-test.mjs # two displays at once, distinct Xwayland displays
node scripts/title-test.mjs # host window title/positionThe smoke test drives the server over stdio exactly like a real client. foot and xterm are
needed for the example clients (not for the server itself).
Machine-specific regression scripts (which drive a particular local app and hardcode its
coordinates) are kept out of the shared tree in scripts/local/ and ignored by git.
Layout
src/index.ts MCP tool definitions
src/display.ts sway lifecycle, sockets, Xwayland detection, host window handling
src/sway.ts swaymsg helpers + window tree flattening
src/input.ts pointer/keyboard routing (XTest vs wlr virtual input)
src/capture.ts grim capture, crop, PNG size
src/util.ts exec helpers, types, PNG header parsing
SKILL.md agent skill (same directory as the server)
docs/ installation guide
scripts/ portable end-to-end tests
scripts/local/ machine-specific tests (gitignored)Known gaps
No audio routing for the nested display (apps use the host PipeWire default).
Clipboard bridging between host and nested display is not implemented.
No OCR / click-text helper yet.
gamescope backend (exact-size game mode, libei injection) is deliberately out of v1.
wlrctl keyboard typehas no named-key action;wtype -kcovers chords.
Available Tools
19 toolsclickClickC
Clicks a mouse button, optionally at absolute output coordinates (or window-relative with relative=true). Uses XTest for X11 targets and the virtual pointer for Wayland targets.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| count | No | ||
| button | No | left | |
| hold_ms | No | How long to hold the button down. Polling UIs (raylib/GLFW, SDL) miss atomic clicks, so this defaults to 100ms. | |
| relative | No | ||
| window_id | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With zero annotations the description carries the full behavioral burden. It usefully discloses the backend split (XTest on X11, virtual pointer on Wayland), but omits the critical behavior of what happens when x/y are omitted (click at current pointer position?) and says nothing about this being an irreversible action dispatched into a live desktop session.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the action and coordinate model front-loaded and no filler. Nothing could be cut without losing signal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Eight parameters, no annotations, no output schema, and low schema coverage demand more than two sentences. Missing: behavior with omitted coordinates, display-id requirement, return/error behavior, and any side-effect disclosure for an action that mutates a real desktop.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 25%, so the description must compensate; it partially does by explaining x/y as absolute output coordinates and 'relative=true' as window-relative. It says nothing about 'count' (repeat clicks), the 'id' display requirement, or 'window_id', leaving half the parameters undocumented in both places.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Clicks a mouse button') and clarifies the coordinate space (absolute output vs. window-relative), which separates it from pointer_move and drag. It never names a sibling or explicitly contrasts with scroll/drag, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to click vs. drag or scroll, no prerequisites (does a display need to be open?), and no when-not-to-use conditions. The only contextual detail is coordinate-space selection, which is parameter semantics rather than usage routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
display_closeClose a nested displayA
Terminates the app and the nested compositor. Pass all=true to close every display.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| all | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the disclosure burden and does add real value by stating the app and nested compositor are terminated. However, it omits reversibility, whether child windows are destroyed, permissions, and idempotency of an already-closed id — meaningful gaps for a destructive tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with the destructive effect front-loaded and the flag usage second. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema and no annotations, so the description should convey outcome details; it names what is terminated but says nothing about the return value or what happens to still-open windows. Adequate but with clear gaps for a destructive multi-param tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50%: 'id' is well documented in the schema, and the description compensates for the bare 'all' boolean by explaining 'all=true to close every display.' Minor gap is no note on behavior when both id and all are supplied.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb ('Terminates') and resource ('the app and the nested compositor'), which implicitly distinguishes it from window_close and pairs with display_open. It doesn't explicitly name a sibling, but the effect is clear enough that an agent can tell this closes a whole display session rather than a single window.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Pass all=true to close every display' gives a conditional usage cue, and 'Optional when exactly one display is open' in the schema hints at the default case. But there's no explicit when-to-use-vs-window_close guidance and no stated exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
display_listList nested displaysA
Lists open displays with their ids, sizes, sockets and app pids.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full load. It does disclose the scope ('open displays' only) and the shape of what comes back, which is real behavioral value, but says nothing about permissions, ordering, or whether the listing is live/snapshot.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the verb and resource, with the return fields appended. Nothing is redundant or padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-param read tool with no output schema, the description compensates by naming the fields that come back. Only minor gaps remain (ordering, whether results are live), which is acceptable at this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate. The baseline for a no-param tool applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Lists') and resource ('open displays') and enumerates the fields returned (ids, sizes, sockets, app pids). It is clearly distinguishable from the mutating siblings display_open and display_close, though it never names them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: 'open displays' hints that this reads current display state, and the sibling names display_open/display_close make the contrast obvious by convention. There is no explicit when-to-use or when-not-to-use statement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
display_openOpen a nested displayA
Starts a nested sway compositor (visible window on the host desktop, with its own Xwayland) and optionally launches an app inside it. Input and screenshots are routed by this server, so no host focus is ever required. Returns the display id and its inner WAYLAND_DISPLAY/DISPLAY values.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Explicit display id (default d1, d2, ...) | |
| app | No | Executable to launch inside the display | |
| cwd | No | ||
| env | No | Extra environment variables for app | |
| args | No | Arguments for app | |
| title | No | Title shown on the host window | |
| width | No | ||
| height | No | ||
| backend | No | How the nested compositor attaches to the host. 'x11' (default when the host has a DISPLAY) gives a Mutter-decorated window that can be moved and retitled; 'wayland' avoids XWayland on the host. | |
| position | No | Host window position (backend 'x11' only) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does substantial work: it discloses that a visible host window with its own Xwayland is created, that input and screenshots are routed server-side so host focus is never needed, and that the response contains the display id and inner WAYLAND_DISPLAY/DISPLAY. It omits lifecycle details — whether the call blocks until the app is up, whether a display must be closed before reuse, and any failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences with the capability stated first, the key operational guarantee (no host focus required) second, and the return values last. No filler or restatement of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 10-parameter, side-effect-heavy tool with no annotations and no output schema, the description covers what is created and what comes back but leaves out the display lifecycle (cleanup via display_close), host prerequisites, and whether concurrent displays with distinct ids are supported. Adequate to call it once correctly, not adequate to manage displays over a session.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 70%, so the schema already documents id, app, env, args, title, backend, and position. The description only reinforces the id and app parameters ('optionally launches an app inside it', 'returns the display id'), and says nothing about cwd, width/height sizing, or the x11-only constraint on position. Baseline 3 is appropriate given the schema does most of the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — starts a nested sway compositor with its own Xwayland — plus the optional app launch, which is far more than the title's 'Open a nested display'. The resource is unambiguous against siblings like display_list and display_close, so an agent can select it without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the agent can infer this is the tool to call first to create a display that screenshot/click/type then target. There is no explicit when-to-use statement, no mention that display_close is the counterpart for teardown, and no note on what to do if an id already exists.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dragDragA
Presses the left button at 'from', moves to 'to' in steps, releases. X11/Xwayland targets only (the virtual pointer protocol cannot hold a button).
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| to_x | Yes | ||
| to_y | Yes | ||
| steps | No | ||
| from_x | Yes | ||
| from_y | Yes | ||
| window_id | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description does carry the core behavior: a left-button press, intermediate motion, and release, plus a platform limitation. It omits the mutation side effects an agent would want to know — whether the button is released on error, how long the drag takes, what return value or exception to expect, and how 'steps' affects timing/smoothness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no filler: the action sequence first, then the platform constraint. Every clause carries information the agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema and no annotations, and 7 parameters with 14% schema coverage leave real gaps — window_id semantics, coordinate frame, and failure behavior are all undocumented. For a mutating input-injection tool this leaves the agent guessing on call correctness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 14% (just 'id'), so the description must compensate, and it partly does by naming 'from' and 'to' and alluding to 'steps'. But window_id is never mentioned, the coordinate space (screen vs. window-relative) is unstated, and 'steps' is left as 'moves to to in steps' without saying whether it is intermediate motion events or a smoothness knob.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a concrete verb sequence (press left button at 'from', move to 'to', release) that fully specifies the drag gesture. This implicitly but clearly separates it from siblings like pointer_move and click, which never hold a pressed button.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states a hard applicability constraint up front: X11/Xwayland targets only, with the reason (the virtual pointer protocol cannot hold a button). However, it names no fallback or alternative for native Wayland sessions, so an agent working there is left without a next step.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
focusFocus a windowB
Focuses a window inside the display (needed before typing into some apps).
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| window_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the entire behavioral burden. Beyond the hint that focus is a prerequisite for typing, it says nothing about failure modes (window id not found, window closed), whether focus raises/restacks the window, whether it steals focus from another display, or blocking behavior. For a state-changing UI operation with zero annotation coverage this is a substantial gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short, front-loaded sentence with a parenthetical that earns its place by explaining why the tool matters. Nothing is padded, though the brevity is partly the cause of the missing parameter and failure-mode detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With two parameters, one required and undocumented, no annotations, and no output schema, the description leaves the agent unable to determine where `window_id` comes from or how to handle the multi-display `id` case. The only thing it fully answers is the high-level purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description never mentions either parameter. Schema coverage is only 50%: `id` is documented as coming from display_open, but `window_id` — the one required parameter — is a bare integer with no source or format guidance anywhere. The description does not compensate for that gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Focuses a window') with the additional scoping detail 'inside the display', which makes the target unambiguous. It does not name the sibling that enumerates window ids (`windows`) or contrast itself with `wait_for_window`, so differentiation from siblings is only partial.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'needed before typing into some apps' gives a concrete condition for reaching for this tool rather than `type`/`key` directly, which is real usage context. There are no explicit exclusions or named alternatives, so it stops short of a full when/when-not statement.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fullscreenToggle fullscreenC
Makes the focused window fullscreen (or restores it).
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| on | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It states the toggle effect, but does not say whether an unfocused window fails, whether this requires a specific display, whether the change is persistent, or what the response is—significant gaps for a state-changing window operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero waste. It is appropriately sized, though the brevity contributes to the under-specification noted in other dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter, state-changing tool with no annotations and no output schema, the description is too thin: it omits the 'on' semantics, the display-id requirement, and the relationship to the 'resize' sibling, leaving the agent with real gaps before calling it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%: the 'id' parameter has a schema description, but 'on' has no description, only a default of true. The description adds no meaning—it never mentions the ability to force a state via 'on' or when 'id' is needed—so it fails to compensate for the gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a clear verb-plus-resource: 'Makes the focused window fullscreen (or restores it).' This lets an agent tell it apart from arbitrary resizing, but it does not name the sibling alternative 'resize' or otherwise explicitly distinguish the two.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The parenthetical '(or restores it)' implies toggle behavior, and the 'focused window' wording implies the entry condition. However, there is no explicit guidance on when to use this versus 'resize', no statement of prerequisites such as a window needing focus, and no mention of what happens when no window is focused.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
keyPress a key combinationA
Presses a key or chord, e.g. 'Return', 'ctrl+alt+Return', 'shift+Tab', 'F10'. Uses xdotool names for X11 targets and xkb keysyms for Wayland targets.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| combo | Yes | ||
| hold_ms | No | How long to hold the chord; polling UIs miss taps between frames. | |
| window_id | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does add real value by disclosing that key names are backend-dependent (xdotool for X11, xkb keysyms for Wayland). It omits other behavioral facts: whether the target window must be focused, how `window_id`/`id` change dispatch, and what errors occur with an invalid combo or no open display.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: purpose/examples first, then the backend naming caveat. No filler and everything is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter, unannotated input tool with no output schema, the description covers key syntax and backend naming but leaves window targeting and failure behavior unexplained. Adequate to invoke correctly in the common case, incomplete for the targeted cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema documents `id` and `hold_ms` but leaves `combo` and `window_id` bare (50% coverage). The description compensates well for the key parameter via syntax examples, but says nothing about `window_id`, leaving the targeting semantics of one parameter undocumented in both places.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Names a specific verb+resource ('Presses a key or chord') and backs it with concrete examples ('Return', 'ctrl+alt+Return', 'shift+Tab'). It implicitly distinguishes itself from the sibling `type` tool by emphasizing single keys/chords rather than text, but never names that alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied through the examples; there is no explicit statement of when to reach for `key` versus `type`, or when `window_id`/`id` targeting is needed. An agent can infer the context but must reason about boundaries itself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pointer_moveMove the pointerA
Moves the pointer to absolute output coordinates. X11 targets use XTest; Wayland targets use the compositor's virtual pointer (unaccelerated).
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| relative | No | Treat x/y as window-relative | |
| window_id | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose platform-specific behavior: X11 uses XTest and Wayland uses the compositor's virtual pointer unaccelerated. It still omits practical behavior such as display requirements, multi-monitor handling, error behavior, and implications of window_id or relative coordinates.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences front-load the core action and then provide relevant platform-specific implementation details. There is no redundant or filler text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a five-parameter tool with no annotations and no output schema, the description covers the core action and platform behavior but leaves key parameter interactions and invocation prerequisites underexplained. It is minimally adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 40%, and the description explains x/y only in terms of 'absolute output coordinates'. It does not compensate for the undocumented window_id parameter or clarify how the relative boolean and window_id interact with the stated absolute-coordinate behavior, creating ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: moving the pointer to absolute output coordinates. The word 'absolute' also distinguishes it from the sibling pointer_move_relative, so an agent can identify the intended operation without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'absolute output coordinates' gives clear context for when this tool is appropriate versus relative movement. However, it does not explicitly name pointer_move_relative or state exclusions such as needing an open display, so it falls short of full alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pointer_move_relativeMove the pointer by a deltaA
Relative pointer motion (compositor-side, unaccelerated). Use this for relative-only consumers: emulated PS/2 mice (86Box, QEMU), games that capture the pointer, or camera controls.
| Name | Required | Description | Default |
|---|---|---|---|
| dx | Yes | ||
| dy | Yes | ||
| id | No | Display id from display_open. Optional when exactly one display is open. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations, so the description carries the full burden. It does disclose two real behavioral traits — motion is compositor-side and bypasses pointer acceleration — which matters for raw-delta consumers. It does not say whether the cursor visibly moves, whether a pointer-lock/capture state is required, or what units the deltas are in.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, with the mechanism stated first and the targeting guidance second. Every clause contributes information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter input-injection tool with no annotations and no output schema, the description covers purpose and audience well but leaves delta units, return behavior, and interaction prerequisites unaddressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 33%: only 'id' is documented in the schema, and the description never explains dx/dy units, sign convention, or the relationship to compositor scale. 'Relative motion' gives partial semantic framing, but for low coverage the description should compensate more than it does.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific action and resource: 'Relative pointer motion', with the qualifier 'compositor-side, unaccelerated' pinning down the mechanism. The 'relative' framing implicitly separates it from the sibling pointer_move (absolute), but it never names that sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete when-to-use guidance with named consumers: emulated PS/2 mice (86Box, QEMU), games that capture the pointer, camera controls. It lacks an explicit exclusion clause ('for absolute positioning use pointer_move'), so the routing is inferred rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
resizeResize the displayB
Changes the display resolution. The host window follows, and screenshots/coordinates use the new size.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| width | Yes | ||
| height | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of explaining behavior. It does disclose two useful side effects the schema cannot: the host window follows the resize, and the screenshot/coordinate space now uses the new size. It says nothing, however, about permissions, failure modes, or whether the change is persisted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, with the mutation stated first and its consequences second. Nothing is wasted, and nothing relevant is buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema and no annotations, so the description must stand alone. It covers the main behavioral consequence well, but it omits units, the id fallback, bounds behavior, and error conditions, for a three-parameter mutation tool with two undocumented numeric parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 33%: only the id parameter is documented. The description does not explain that width and height are in pixels and does not mention the optional id or its default of 'the single open display.' The only clue is the word 'resolution,' which only implicitly covers the units.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: it changes the display resolution. That is clear enough to separate it from siblings like fullscreen or display_open, but the description never explicitly names those alternatives, so the differentiation is only implicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It does not say when to prefer this over fullscreen, or what preconditions apply, such as whether a display must already be open or whether resolutions are constrained by the host. The note about the window following is a side effect, not usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotScreenshot the nested displayA
Captures the composited frame (grim) as PNG - exact pixels, palette already resolved, includes Wayland and X11 clients. Coordinates in the image are the same coordinates used by click/pointer_move.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| scale | No | Output scale factor | |
| width | No | ||
| height | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does well by disclosing the capture mechanism (grim), output format (PNG), pixel fidelity, palette resolution, and inclusion of both Wayland and X11 clients. It still leaves out return-shape details such as whether the PNG is returned inline or as a file path, and does not discuss blocking behavior or multi-display requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two efficient sentences with the core behavior front-loaded and no wasted text. The coordinate-system note is also placed usefully at the end.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a six-parameter screenshot tool with no annotations, no output schema, and low schema coverage, the description covers purpose and some behavior but does not explain how x/y/width/height act as crop or region parameters or how scale affects output. It is adequate but leaves meaningful gaps for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33%, and the description does not explain the x, y, width, height, or scale parameters beyond noting that image coordinates match click/pointer_move coordinates. The schema already documents id and scale, so the description compensates for very little of the undocumented parameter surface.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: it captures the composited display frame as PNG, with exact pixels and both Wayland and X11 clients included. It clearly describes a full composited display capture rather than a window capture, though it does not explicitly name screenshot_window as the alternative.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the mention of the composited frame and all clients, suggesting this is for capturing the whole display. However, there is no explicit when-to-use guidance, no exclusions, and no direct routing to screenshot_window for window-specific captures.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshot_windowScreenshot one windowC
Captures the window's rectangle from the composited frame.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| scale | No | ||
| window_id | Yes | Window id from the windows tool |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry behavioral disclosure. It mentions capturing from the composited frame, which adds some context, but it does not state whether the window must be visible/focused, what permissions are needed, failure behavior, or what the capture returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no filler. "from the composited frame" is useful context and nothing is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a screenshot tool with no output schema and no annotations, the description is incomplete. It does not explain the return value, output format, scale behavior, or failure modes, all of which an agent needs to invoke the tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%; window_id and id are documented in the schema, but scale is undocumented. The description adds no meaning about scale, display selection, or how the parameters interact, so it does not compensate for the remaining gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb and resource: capturing a window's rectangle from the composited frame. It is clear what the tool does, but it does not explicitly distinguish this from the sibling screenshot tool that likely captures displays or full screens.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus screenshot, windows, or wait_for_window. The phrase "window's rectangle" implies it is for window-specific captures, but no alternatives or conditions are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scrollScrollC
Scrolls vertically (dy) and horizontally (dx) at the given position or the current pointer.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| dx | No | ||
| dy | No | ||
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| window_id | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It mentions vertical and horizontal scrolling and position versus current pointer, but omits whether focus/display/window is required, what happens without an active pointer, units of dx/dy, and whether scrolling is immediate or animated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no redundant or filler content. It states the action and axes efficiently, though its brevity contributes to the missing context noted elsewhere.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With six parameters, no annotations, no output schema, and low schema description coverage, the description is not complete enough for reliable invocation. It omits window/display targeting, coordinate semantics, scroll units, and prerequisites that an agent needs to call this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 17%, with only 'id' documented in the schema. The description adds meaning for dx/dy as horizontal/vertical scroll amounts and implies x/y are coordinates, but it does not explain window_id, units, defaults, or the interaction between position and pointer.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Scrolls vertically (dy) and horizontally (dx)'. It also clarifies the scroll target context ('at the given position or the current pointer'), though it does not explicitly distinguish itself from sibling pointer/window tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus siblings like pointer_move, click, or focus, nor any prerequisites such as whether a display or window must be active. The only usage hint is implicit: scrolling can occur at a given position or the current pointer.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
typeType textA
Types text into the focused (or given) window. X11/Xwayland targets use XTest, Wayland targets use wtype. Do not mix the two paths in one session: keymap uploads can garble XTest text.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| text | Yes | ||
| window_id | No | Focus this window first |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does real work: it discloses the two backend paths (XTest for X11/Xwayland, wtype for Wayland) and a concrete failure mode (keymap uploads garbling XTest text). It stops short of stating timing, Unicode/character support, or what happens when no window has focus.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: purpose first, then mechanism selection, then the hazard warning. Front-loaded and zero filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter, no-output-schema, no-annotation tool, the description covers targeting, backend selection, and a critical cross-session pitfall. Missing only error/edge behavior (no focused window, unsupported characters), which is minor at this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 67%: `id` and `window_id` are documented in the schema, and the phrase "focused (or given) window" usefully reinforces that targeting is optional. The required `text` parameter has no schema description and gets no clarification here (format, newline handling, size limits), leaving a small gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb and resource ("Types text into the focused (or given) window"), which is enough to separate it from the keyboard-shortcut sibling `key`. It does not explicitly name `key` as the alternative for non-text input, so sibling differentiation is implicit rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage context is implied: use this to enter text, targeting the focused window by default or a specific one. The one explicit rule — "Do not mix the two paths in one session" — is a constraint, not a when-to-use guide, and the description never says when to prefer `key` or `focus` instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
waitWaitA
Sleeps for the given milliseconds (use sparingly, e.g. after launching an app).
| Name | Required | Description | Default |
|---|---|---|---|
| ms | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden; it does disclose that the call blocks for the duration and hints at the cost via "use sparingly". It omits the upper bound (60s cap, only in the schema) and says nothing about whether the delay blocks the session or other calls, which matters for a sleep tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with the core action first and the guidance in a tight parenthetical. Nothing is wasted and no reading is required beyond the first clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter sleep with no output schema and no annotations, the description covers what it does and when it is appropriate. The missing piece is the relationship to wait_for_window, which would prevent a badly chosen polling sleep.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
One parameter with 0% schema description coverage, so the description must compensate. It clarifies the unit (milliseconds), which the parameter name alone only implies, but adds nothing about the 0–60000 range or what happens at the boundary.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: sleeps for a given number of milliseconds, which is unambiguous about what the call does. It does not distinguish itself from the sibling wait_for_window, which is the closest functional alternative an agent will consider.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Use sparingly, e.g. after launching an app" gives one concrete when-to-use case plus a caution, which is more than nothing. However, it never names wait_for_window — the sibling that should be preferred when the agent is waiting for a window to appear — so the key routing decision is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
wait_for_windowWait for a windowA
Waits until a window whose title, app_id or class matches the regex exists. Returns the window.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| pattern | Yes | ||
| timeout_ms | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the behavioral burden, and it does disclose that the tool blocks until a matching window exists and then returns that window. It does not explain timeout behavior, what happens if no window appears, or how multiple matches are resolved. Those omissions keep it at minimum viable transparency rather than strong disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two compact sentences with the primary behavior front-loaded. Every clause adds useful information about matching and return behavior, with no repetition or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a wait tool with no annotations and no output schema, the description covers the core matching fields and return value. It still omits critical context such as timeout behavior, error behavior when the regex never matches, and whether multiple windows can match. This is adequate but not complete for reliable invocation in edge cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33%, so the description needs to compensate. It does explain pattern semantics by stating that the regex matches title, app_id, or class, while id is covered by the schema. However, timeout_ms is not described in either place, so parameter semantics remain partially incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: it waits for a window matching a regex and returns it. It also names the three matchable fields, title, app_id, and class, so it is more specific than a generic wait. Sibling differentiation is mostly implied rather than explicit, since it does not directly contrast with windows, wait, or focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the condition "until ... exists," which suggests calling it when a window may not yet be present. However, it does not explicitly say when to prefer it over windows, wait, or focus, and it gives no exclusions or prerequisites. This is adequate but leaves routing to the agent's inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
window_closeClose a windowB
Asks the compositor to close a window (as if clicking its close button).
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. | |
| window_id | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden, and it does disclose the important semantics: this is a polite request to the compositor equivalent to a close-button click, implying the app may prompt or refuse. It says nothing about whether the call blocks until the window disappears, how refusal or a missing window is reported, or whether it is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence with a high-value parenthetical and zero filler. Nothing could be removed without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter GUI action with no annotations and no output schema, the description covers what the action does and its graceful-close nature, which is the core need. It is still incomplete on the undocumented 'window_id', on whether the call waits for the window to disappear, and on failure behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 50%: the display 'id' is described in the schema, but 'window_id' is an undocumented integer in both schema and description. The description adds no parameter meaning at all, so it fails to compensate for the coverage gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('close a window') and adds the mechanism ('asks the compositor... as if clicking its close button'), which separates it from display_close in spirit. It doesn't explicitly name display_close or a force-kill sibling, so the differentiation is implied rather than stated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance: nothing says whether to prefer this over display_close, whether a window must be focused first, or what to do if the app refuses to close. The parenthetical hints that this is a graceful request rather than a forced termination, but that is behavioral detail, not usage routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
windowsList windowsA
Lists windows inside the display: id, title, app_id/class, shell (xwayland or xdg_shell), rect, focus and pids. Rects are in output coordinates, matching screenshots.
| Name | Required | Description | Default |
|---|---|---|---|
| id | No | Display id from display_open. Optional when exactly one display is open. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations at all, the description carries the full burden, and it does add real context by disclosing the returned fields and clarifying that rects are in output coordinates matching screenshots — a genuinely useful behavioral note. It still omits whether a display must be open, result ordering, whether hidden/minimized windows are included, and whether the call blocks or is instant.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences with zero filler: the first defines the payload, the second resolves the most likely ambiguity (coordinate space) up front. Nothing repeated from the schema or title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description compensates by enumerating the returned fields and pinning down the coordinate system — enough for an agent to consume results correctly. Only minor gaps remain around display preconditions and result ordering.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single optional 'id' parameter, and the schema already explains the 'optional when exactly one display is open' fallback. The description adds nothing about the parameter, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Lists windows inside the display') and then enumerates exactly what each entry contains (id, title, app_id/class, shell, rect, focus, pids). An agent can tell this is an enumeration tool rather than an action tool, though it doesn't explicitly contrast itself with siblings like wait_for_window or screenshot_window.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: the field enumeration suggests 'call this to discover windows before acting on them', but there is no explicit when-to-use, when-not-to-use, or alternative named. The relationship with wait_for_window (polling vs one-shot listing) is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
19 tool updates
v0.1.0- First observed
click - First observed
display_close - First observed
display_list - First observed
display_open - First observed
drag - First observed
focus - First observed
fullscreen - First observed
key - First observed
pointer_move - First observed
pointer_move_relative - First observed
resize - First observed
screenshot - First observed
screenshot_window - First observed
scroll - First observed
type - First observed
wait - First observed
wait_for_window - First observed
window_close - First observed
windows
TDQS
Scored across 19 tools
Every tool targets a distinct action or resource: display lifecycle (open/list/close), capture (full frame vs window rect), pointer motion (absolute vs relative), button actions (click/drag/scroll), text input vs key chords, and window management. The descriptions explicitly resolve the few near-neighbors (screenshot vs screenshot_window, pointer_move vs pointer_move_relative, type vs key) by stating when each path applies.
The set mixes conventions: noun_verb prefixed lifecycle tools (display_open/display_list/display_close), bare verbs/nouns (click, drag, scroll, type, key, focus, windows, screenshot), and multiword forms like screenshot_window, pointer_move_relative, wait_for_window. It remains readable and somewhat grouped, but there is no single predictable pattern an agent can rely on.
Nineteen tools is on the heavy side but each maps to a real capability in a rich domain (display lifecycle, screenshots, window management, four input modalities, timing). No obvious duplicates or filler, though a couple of tools (wait, fullscreen) are modest additions to an otherwise dense input surface.
The surface covers the full lifecycle: create/list/close displays, capture, enumerate/focus/close/wait for windows, and drive pointer, buttons, scroll, text, and keys, plus resolution changes. Minor gaps remain—no per-window move/resize (only display resize), and drag is documented as X11-only, limiting Wayland interaction workflows.
Maintenance
Related MCP Connectors
Cloud desktops for AI agents: create a Linux computer, see its screen, click, type and run tasks.
- TabfleetOAuthcom.tabfleet
Launch, inspect, control, and share isolated cloud browsers for your agents.
Persistent Linux computers for AI agents: desktop, signed-in browser, vault, apps and shell.
Remote Linux boxes for coding agents: Docker, a browser, screenshots, logs, human takeover.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceEnables AI agents to automate Linux desktop GUI by launching and interacting with Wayland applications in isolated virtual KWin sessions, or connecting to live desktops for collaborative automation.155 PyPI76MIT
- AlicenseAqualityBmaintenanceEnables AI agents to operate a real GNOME Wayland desktop through accessibility-tree widget actions, pointer/keyboard input, OCR, window management, and screen capture, optionally on a private headless session.3331 PyPI11Apache 2.0
- AlicenseAqualityBmaintenanceEnables an agent to see and operate a GNOME Wayland desktop like a human, including screenshots, mouse/keyboard control, accessibility tree access, OCR, clipboard, and audio listening.13MIT
- AlicenseNot gradedqualityBmaintenanceGives each agent its own isolated nested Hyprland desktop with a private seat, clipboard and accessibility bus, so it can see the screen and drive a mouse, keyboard, touchpad gestures, windows and apps without touching the user's pointer, focus or windows. Exposes screenshots, zoom, OCR, accessibility-tree control, window management and batched actions as MCP tools, and can optionally act read-only, or with input, on the real desktop.2MIT