hyprcu
Provides desktop control for Hyprland, enabling AI agents to interact with windows, workspaces, input (pointer/keyboard), screenshots, and application launching through Hyprland IPC.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@hyprcutake a screenshot of the file browser window"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
hyprcu
Hyprland computer use. Dialog-free desktop control for AI agents: an MCP
server and a CLI over the same primitives. A fork of
hypruse (MIT, IlyasKhallouki) — see Lineage
with the trust, journal, safety-beacon and CLI layers removed, keeping the
Wayland/Hyprland primitives intact. App knowledge and tooling live in docs/ and tools/.
What's kept (byte-identical to upstream)
module | what |
| raw |
| AT-SPI tree via |
| Hyprland socket2 listener → |
| IPC layer; handles both hyprlang and Lua-dispatch Hyprland |
| as upstream |
| the MCP server, unmodified |
Related MCP server: hypruse
What's replaced with no-op stubs
trust.py, journal.py, safety.py, skill.py — same public names, every guard passes,
every log call is a no-op. server.py didn't need a single edit.
What's removed
waybar/, packaging/, and upstream's init wizard, journal/replay
commands and skill installer.
CLI (shell verbs) — kept
Every MCP tool is also a shell verb, ~150ms per fresh process, for agents that only have bash (Codex, Claude Code in a terminal, scripts):
hyprcu # no args = MCP server on stdio
hyprcu desktop # one line per monitor/workspace/window
hyprcu screenshot --window 0x… # prints path + {"geometry","scale",…}
hyprcu hypr focus_window 0x…
hyprcu pointer click 800 60
hyprcu keyboard type "hello" --window 0x…
hyprcu sequence '[{"op":"keyboard","action":"type","text":"x"},…]'
hyprcu doctor # check binaries + session
hyprcu stop # kill any running server/verbExit codes: 0 delivered · 1 error · 2 usage · 4 nothing to act on.
Tests
uv run pytest tests/ --ignore=tests/test_e2e.py → 359 passed, 54 xfailed.
The xfails are enumerated in tests/removed_guard_tests.txt; each asserts
that a guard refuses something. They are strict, so a guard silently
coming back would fail the suite.
Run
uv sync
uv run python -m hyprcu # MCP server on stdioTools: desktop, screenshot, zoom, ui, marks, binds, wait_for, pointer, keyboard, click_ui, hypr, launch, use_bind, sequence.
Measured on this machine (2026-09-20)
launch foot→ 167ms, returns address (event-driven, no sleep)sequencetype+enter → 627ms, one round-tripclose_window→ 3ms, confirms destroy eventuion Strata: 1 element (sidebar toggle only — custom-drawn list)uion Slack: 0 elements (Electron, needs--force-renderer-accessibility)
The a11y tools are only as good as the apps' trees. On this desktop, today,
they're mostly empty. sequence, launch, wait_for and the unified
pointer are the real gains.
What hyprcu adds (the parts that are ours)
pick.py — natural-language window targeting. Every window argument
(hypr, pointer, keyboard, screenshot, ui, click_ui…) accepts an address, a
class/title substring, or a description. Resolution: exact address → unique
substring (free) → kev choice (~200ms, local). Ambiguous substrings are also
tie-broken by kev. Below KEV_GATE (0.5) it fails with "the app may not be
open — check desktop() or launch it", which has been right every time so far.
Results carry [kev: 99% in 229ms] so you can see when it was used.
hyprcu hypr focus_window "the file browser"
hyprcu keyboard type "hello" --window "the shell on workspace 2"Window-relative clicks. pointer takes window + x_pct/y_pct
(0.0–1.0); the window is focused first and the fraction is mapped to its
current geometry, so the click survives moves and resizes. CLI:
hyprcu pointer click --in "Strata" --at 0.053 0.23.
journal.py — training log. Acting tools append one JSONL row to
~/.local/share/hyprcu/actions.jsonl (HYPRCU_LOG=0 disables): args,
the window list at the time, result, and — when kev chose — query and
probability. Rows are unlabelled; a correct field is meant to be added
later before anything is trained on them.
Requires a Jev-compatible server at KEV_URL (default kev-4b on :8009, see
pi-omarchy-computer-use/kev-serve.sh). Without one, substring and address
targeting still work; descriptions fail with an explicit message.
No built-in judgement (2026-09-20)
hyprcu does what it's asked. Checks and controls belong in the caller, not here. Removed from upstream's acting tools, beyond the trust layer:
was | now |
| runs every step; |
| effectively unbounded |
| caller's value |
| caller's value |
The event stream still serves wait_for steps inside a sequence (so an
event between steps isn't missed) — that's plumbing, not a guard.
tests/test_no_guards.py pins this contract.
Trust/approval language audit (2026-09-20)
Verified against the live MCP wire, not just the source:
No code path can raise a refusal:
grep "raise TrustError"→ 0 hits outside the stub's class definition.Server
instructions(what the model reads at connect): 0 gating terms.Tool descriptions: the three
allow_auth=true overrides the refusal…sentences, "panic-kill guarantees", and "Refused while HYPRCU_CONFINE" removed. Remaining "confirm" is "screenshot to confirm the click worked".allow_authis still an accepted boolean on pointer/keyboard/click_ui — it flows only into no-op stubs, FastMCP emits it with no description, and 8 upstream tests pass it. Left in to keep server.py logic identical to upstream; harmless.HYPRCU_READONLY=1still works as an opt-in "observe only" mode (hides acting tools). Nothing sets it by default.
Lineage
hyprcu is a fork of hypruse by
IlyasKhallouki (MIT). The Wayland/Hyprland primitives — wire.py (virtual
pointer), a11y.py, events.py, hyprctl.py, input.py, screenshot.py
and the MCP server.py — are his work, kept close to upstream so fixes can be
merged. hyprcu removes the trust/journal/safety layers and every built-in
refusal, and adds kev-based natural-language targeting, window-relative
coordinates, and a training log. Upstream reference checkout: ~/Work/hypruse.
Available Tools
14 toolsbindsA
The user's own Hyprland keybinds: combo, action, arg, and a
description when the config provides one. This is how the desktop's
owner drives it: to perform one of these workflows, call use_bind
with the combo (it runs the bound action). An action of lua means the
bind is a closure in a Lua Hyprland config, which nothing can run from
outside: read its description and do the same thing with hypr or
launch. NOTE: the keyboard tool canNOT trigger these compositor
binds (synthetic keys reach apps, not Hyprland's bind matcher), so do
not try to press them.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that this is a read-only listing of binds, that `lua` actions are closures that cannot be run externally, and that synthetic keys from the `keyboard` tool will not trigger compositor binds. It does not explicitly state that the tool returns a list, but the output schema exists and the description's framing as 'the user's own keybinds' implies a listing. The behavioral caveats about `lua` and `keyboard` are valuable beyond what any schema would show.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single dense paragraph that front-loads the core purpose and then adds routing guidance. Every sentence earns its place: the first defines the resource, the second explains how to use it, the third handles the `lua` edge case, and the fourth warns against the `keyboard` tool. It is slightly long but not bloated, and the structure is logical. A small formatting improvement (separating the warning) would make it a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a zero-parameter listing tool with an output schema, the description covers the essential context: what the data is, how to act on it, and what not to do. It does not describe the exact return format, but the output schema exists and the description's mention of fields (combo, action, arg, description) aligns with that. The edge case of `lua` actions is handled. The only minor gap is not explicitly stating that the tool returns a list of all binds, but that is inferable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema is trivially complete (100% coverage). The description adds context about what the returned data means (combo, action, arg, description) and how to use it, which is more than a baseline 4 would require. Since there are no parameters to document, the description's job is to explain the data model and usage, which it does.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that this tool exposes the user's own Hyprland keybinds (combo, action, arg, description) and explicitly frames it as a read-only listing of the desktop owner's workflows. It distinguishes itself from siblings by explaining that these are compositor-level binds that the `keyboard` tool cannot trigger, and it names `use_bind` as the way to execute them. The verb 'binds' is a noun here, but the description makes the resource and its role unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: to perform one of these workflows, call `use_bind` with the combo. It also provides a when-not-to-use exclusion: the `keyboard` tool cannot trigger these binds, so do not try to press them. It even handles the `lua` action edge case, telling the agent to read the description and use `hypr` or `launch` instead. This is exemplary routing behavior.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
click_uiA
Click a control by its accessible NAME, or by a mark number from
the last marks capture, in ONE call: the exact coordinate comes from
the accessibility tree, the window is focused first, and the click goes
through the real pointer (visible cursor),
so no screenshot and no pixel estimation is spent. Pass exactly one of
name (matched against window's controls, exact accessible name
preferred, substring otherwise) or mark. An ambiguous name returns
the candidates instead of guessing: disambiguate with index (0-based
into that list) or a more specific name. Falls back with a note when
the app exposes no tree (use screenshot + zoom + pointer then).
then ('desktop'|'screenshot'|'ui'|'none') appends the result;
'ui' shows the click's effect on the controls in the same call.
| Name | Required | Description | Default |
|---|---|---|---|
| mark | No | ||
| name | No | ||
| then | No | none | |
| index | No | ||
| button | No | left | |
| double | No | ||
| window | No | ||
| allow_auth | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries behavioral disclosure. It reveals that the window is focused first, the click uses the real pointer with a visible cursor, coordinates come from the accessibility tree, ambiguous names return candidates instead of guessing, and fallback behavior is explicitly noted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense and well-structured, front-loading the core mechanism before detailing options. Every sentence adds necessary guidance; there is no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of annotations and an 8-parameter schema, the description is largely complete: it covers the main invocation paths, disambiguation, fallback strategy, and post-click behavior. The only notable omissions are the semantics of `button`, `double`, and `allow_auth`, which would be useful for full autonomy.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It thoroughly explains `name`, `mark`, `index`, `then`, and partially `window`. However, `button`, `double`, and `allow_auth` are not described beyond their schema defaults, leaving some gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a precise verb and resource: click a control by accessible name or mark number through the accessibility tree. It clearly differentiates from sibling tools by emphasizing no screenshot, no pixel estimation, and one-call execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage rules: pass exactly one of `name` or `mark`, disambiguate ambiguous names with `index` or a more specific name, and fall back to screenshot + zoom + pointer when no accessibility tree exists. It also explains the `then` parameter's options and their effects.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
desktopA
Semantic desktop snapshot: monitors, workspaces, windows (address,
class, title, at + size in global coords), active window, cursor,
and layers: launchers (wofi/rofi), bars, notification popups, and
on-screen keyboards are NOT windows and appear only there, with a
best-effort kind and global geometry you can screenshot by region or
click into. A listed layer is one the compositor tracks, not one you
can see: it may be transparent or dormant, so screenshot when
visibility matters. Call first; act on the addresses it returns.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full burden. It reveals important behavioral nuances: layers are not windows and may be invisible, and the snapshot returns geometry for screenshotting or clicking. It also notes that 'a listed layer is one the compositor tracks, not one you can see,' which is critical for correct interpretation. This is rich, non-obvious context beyond what a schema could convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise but information-dense. It front-loads the core purpose ('Semantic desktop snapshot') and then provides necessary details about layers and action points without fluff. Every sentence adds value: the layer clarification, the visibility caveat, and the directive to call first. The structure is logical, progressing from what, to nuances, to usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (snapshot of multiple desktop elements) and the absence of annotations, the description provides sufficient detail for correct invocation and interpretation. It explains the distinction between windows and layers, mentions best-effort 'kind' for layers, and advises on screenshot when visibility matters. The output schema likely details the return format, so the description complements it well.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the description adds value by explaining what the output contains (the snapshot details). Even though there's no parameter to document, the description clarifies the structure of the returned data, which is essential for the agent. Since there are no parameters, the description complements the lack of input schema with domain context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that this tool takes a semantic snapshot of the desktop environment, enumerating specific entities (monitors, workspaces, windows, active window, cursor) and their properties (address, class, title, geometry). It distinguishes itself from siblings like 'screenshot' and 'click_ui' by emphasizing the returned addresses and the semantic layer, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit instructions are given on when to use this tool: 'Call first; act on the addresses it returns.' It also provides guidance on handling layers, noting that they may be transparent or dormant, and advises to screenshot when visibility matters, which helps the agent decide between using this snapshot tool versus a screenshot tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hyprA
Window/workspace ops over IPC (instant, no vision).
action='workspace' (workspace: number/name/'special:name') |
'focus_window' (target: address) | 'move_window' (target + workspace,
silent) | 'close_window' (target) | 'fullscreen' (target?) |
'toggle_floating' (target?) | 'dpms_on' / 'dpms_off' (display power;
screenshots cannot capture a dark display — desktop() reports it and
screenshot errors with a pointer here). then ('desktop'|'screenshot'|'ui'|'none')
appends the result to this call.
| Name | Required | Description | Default |
|---|---|---|---|
| then | No | none | |
| action | Yes | ||
| target | No | ||
| workspace | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It discloses that operations are instant, that move_window is silent, that dpms_on/off changes display power, and that then appends output to the same call. It also flags a concrete downstream failure mode (screenshot/desktop on a dark display), which is valuable transparency beyond the bare schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose is front-loaded and every subsequent phrase adds semantic value. The pipe-separated action list is dense but compact, containing no filler or repetition. It packs a complete operational spec into a few lines.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no annotations and no schema parameter descriptions, the description gives a complete operating picture: actions, parameter meanings, optionality, chaining behavior, and a cross-tool caveat. Since an output schema exists, return-value details are not required here. Nothing essential is missing for an agent to invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must define the parameters, and it does thoroughly. It enumerates valid action values, specifies workspace as number/name/'special:name', marks target as an address with optional usage where relevant, and explains the then options and their effect. This fully compensates for the empty schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens by naming the resource (window/workspace) and mechanism (IPC, instant, no vision), then enumerates each supported action with its arguments. This makes the tool's scope immediately identifiable and distinguishes it from visual/graphical siblings like screenshot and desktop.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives clear context for when to use the tool: fast window/workspace operations over IPC without vision, and it explicitly warns that dpms_off makes screenshots/desktop fail on a dark display. It does not explicitly name sibling alternatives with a 'use X instead' formula, but the guidance is strong enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
keyboardA
Keyboard to the focused app. action='type' (text, unicode-safe) |
'key' (keys combo: 'ctrl+shift+t', 'esc', 'F5'; aliases
enter/esc/tab/backspace/pgup/pgdn/arrows, else XKB keysyms). Pass
window (an address from desktop) to focus that window first, so
keystrokes land in the intended app rather than whatever currently
holds focus. This drives shortcuts the focused application handles
(ctrl+t, ctrl+l). It does NOT trigger Hyprland's own keybinds
(super+...): those go through use_bind, and workspace/window actions
through hypr. then ('desktop'|'screenshot'|'ui'|'none') appends the
result to this call.
| Name | Required | Description | Default |
|---|---|---|---|
| keys | No | ||
| text | No | ||
| then | No | none | |
| action | Yes | ||
| window | No | ||
| allow_auth | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It explains focus targeting, the distinction between app shortcuts and compositor keybinds, the `then` result-appending behavior, and unicode-safety for text. It omits the meaning of `allow_auth`, leaving a minor transparency gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense and information-rich with no filler, front-loading the core action modes before focus and routing guidance. It could be slightly more structured, but every sentence adds necessary value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a multi-parameter input tool with no annotations and 0% schema coverage, this covers most invocation essentials: action formats, focus handling, routing exclusions, and `then`. The unexplained `allow_auth` parameter and lack of explicit text/keys pairing instructions prevent a perfect score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It explains `action` values, `keys` formats and aliases, `window` semantics, and `then` options. However, `allow_auth` is not described, and the relationship between `action` and `text`/`keys` is only implicit.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it sends keyboard input to the focused app and differentiates the two action modes ('type' vs 'key'). It also explicitly names what it is not for (Hyprland keybinds), distinguishing it from siblings like use_bind and hypr.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use guidance: application-level shortcuts like ctrl+t and ctrl+l. It also provides clear when-not-to-use instructions, routing Hyprland keybinds to use_bind and workspace/window actions to hypr, plus advice to pass `window` for correct focus.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
launchA
Run command via Hyprland exec. Optional workspace placement
(silent, works even for single-instance apps like browsers, whose
window gets moved after it appears) and wait_s (1-30, default 8;
raise for slow apps). Returns the new window's
address/class/title/workspace, or a timeout note.
| Name | Required | Description | Default |
|---|---|---|---|
| wait_s | No | ||
| command | Yes | ||
| workspace | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses behavioral traits like workspace placement behavior for single-instance apps and timeout, but could mention error cases or permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no fluff. First sentence states purpose, second explains optional parameters and return value. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given low complexity and presence of an output schema, the description is complete. It summarizes return fields and timeout note, sufficient for the agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description adds complete meaning for each parameter: command, workspace placement, wait_s range and default. Fully compensates for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a command via Hyprland exec, with specific verb and resource. It also distinguishes itself from sibling tools (desktop, hypr, etc.) by focusing on launching commands.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use, including optional workspace placement and wait_s parameter. However, it does not explicitly state when not to use or mention alternatives, though siblings are distinct.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
marksA
Set-of-Marks capture: a screenshot of the window WITH its accessible
controls drawn as numbered red marks, plus a JSON legend mapping each
number to the control's role, name, current value, and exact global
click point. One glance replaces the estimate-zoom-estimate loop for
every control the accessibility tree knows: read the number off the
image and call click_ui(mark=N) (or pointer at the legend's x,y).
window is an address from desktop (default: focused); name filters
the marked controls. Falls back to the plain legend when ImageMagick is
not installed, and to a fall-back-to-vision note when the app exposes
no tree (then use screenshot + zoom).
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| window | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the capture behavior, reliance on accessibility tree, fallback to plain legend when ImageMagick is absent, and a note when no tree exists. It does not mention destructive actions, rate limits, or authentication, but these are not critical for a read-like capture tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is moderately long but all sentences serve a purpose: core functionality, usage advice, fallback conditions. It is front-loaded with the primary action. Minor redundancy (e.g., 'Falls back...') could be more concise, but overall well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of fallbacks (ImageMagick, no tree) and the presence of an output schema (return values not needing further explanation), the description covers all necessary context: parameters, output (JSON legend), and use case. It references sibling tools appropriately.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds meaning: `window` is 'an address from desktop (default: focused)' and `name` 'filters the marked controls'. It does not specify the exact format of the window address, but provides enough context for the agent to use parameters correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool captures a screenshot with numbered red marks and a JSON legend mapping each mark to control details. It uses the specific verb 'capture' and resource 'Set-of-Marks', and distinguishes from siblings like 'screenshot' and 'ui' by highlighting marks and legend.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises when to use the tool: to replace the 'estimate-zoom-estimate loop' for controls in the accessibility tree. It provides follow-up actions (using `click_ui(mark=N)` or `pointer`), and specifies fallbacks for missing ImageMagick or no accessibility tree, with an alternative (screenshot + zoom).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
pointerA
Mouse. action='move' (x,y) | 'click' (optional x,y first; button left/right/middle; double=true) | 'drag' (x,y → to_x,to_y holding button) | 'scroll' (scroll_dy notches, positive = content down; optional x,y first).
Coordinates are GLOBAL logical pixels by default. Pass window (address,
class/title substring, or a description like "the file browser") plus
x_pct/y_pct in 0.0–1.0 to click RELATIVE to that window instead: (0.5,0.5)
is its centre, (0.1,0.05) near its top-left. The window is focused first.
This is the robust form — it survives the window moving or resizing.
then appends the result to this call so you skip a round-trip: 'desktop'
a fresh snapshot, 'screenshot' a stable capture, 'ui' the focused window's
elements with current values, 'none' (default) nothing.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| then | No | none | |
| to_x | No | ||
| to_y | No | ||
| x_pct | No | ||
| y_pct | No | ||
| action | Yes | ||
| button | No | left | |
| double | No | ||
| window | No | ||
| to_x_pct | No | ||
| to_y_pct | No | ||
| scroll_dx | No | ||
| scroll_dy | No | ||
| allow_auth | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden and handles it well: it discloses coordinate defaults, window focusing, scroll direction sign, double-click support, and exactly what `then` returns. It does not explain allow_auth or potential side effects of authentication, but core mouse behavior is transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a compact action grammar, then expands coordinate semantics and `then` behavior in clear, short sections. It is dense but not padded; every sentence adds operational value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex 16-parameter tool with no annotations and no schema descriptions, the description covers the main actions and coordinate modes well, and an output schema exists to cover return values. It is not fully complete because to_x_pct/to_y_pct, scroll_dx, and allow_auth remain unexplained, leaving several schema properties opaque.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, and the description compensates by explaining most parameters: action values, x/y, to_x/to_y, button, double, scroll_dy, x_pct/y_pct, window, and then. It leaves to_x_pct/to_y_pct, scroll_dx, and allow_auth undocumented, which is a meaningful but secondary gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description defines the tool as a mouse control with a clear action grammar: move, click, drag, and scroll, including coordinate semantics. It is clearly a pointer/mouse tool, but it does not explicitly differentiate itself from sibling input tools like click_ui or keyboard.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives strong within-tool guidance: global pixels by default versus window-relative percentages, explains that the window is focused first, and calls the relative form robust because it survives window movement or resizing. However, it never says when to choose pointer over sibling tools such as click_ui or keyboard, so cross-tool selection guidance is implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotA
Capture the focused monitor, a window (window: "active" or an
address from desktop, cheapest for reading one app), or a region
"x,y,WxH". Returns the image (or a file path to read) + JSON metadata
with geometry/scale for pixel→global mapping. scale 0.1-1.0:
optional deliberate downscale, usually leave unset. stable=true
waits (up to 2s) until two consecutive frames match, so a capture
right after an action is not taken mid-animation; metadata gains
stable. Captures are fast JPEG by default; lossless=true returns
PNG for pixel-exact work. Before clicking a small control, follow with
zoom at the estimated point.
| Name | Required | Description | Default |
|---|---|---|---|
| scale | No | ||
| region | No | ||
| stable | No | ||
| window | No | ||
| lossless | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fully discloses behavior: default JPEG, lossless PNG option, stable wait up to 2s, scale downscale, and metadata details. Contradicts no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph, front-loads main verb and resource. Slightly dense with multiple details but no wasted words. Could be split for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers input parameters comprehensively, mentions output format (image/path + JSON metadata), and provides usage advice. Output schema exists, so return details are not needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description explains all 5 parameters: window, region, scale, stable, lossless. Each parameter's purpose and effect are clearly described beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool captures a monitor, window, or region, and specifies output format. It distinguishes from siblings like 'zoom' by mentioning its purpose after screenshot.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context on when to use 'window' (cheapest for reading one app) and advises to follow with 'zoom' for clicking small controls. Lacks explicit when-not-to-use scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
sequenceA
Run an ordered list of actions in ONE call, so a click/type/enter
micro-sequence costs one round-trip instead of several. Each step is
{"op": "pointer"|"keyboard"|"click_ui"|"hypr"|"wait_for", ...that tool's
args}, e.g. [{"op":"pointer","action":"click","x":800,"y":60},
{"op":"keyboard","action":"type","text":"hello","window":"0x.."},
{"op":"keyboard","action":"key","keys":"enter"}]. Every step runs, in
order, regardless of what the desktop does in between; a step that
raises stops the run and reports which one. To type into a specific
window reliably give that keyboard step a window= (address, substring
or description; it focuses first). Opt-in stop_on_change=true aborts
between steps if a window opens/closes/moves or the workspace switches
unexpectedly — useful when a dialog might appear, but it also fires on
changes you intended (Ctrl+T opening a tab), so it is off by default.
then observes the final state ('desktop' default, 'screenshot', 'ui',
'none').
| Name | Required | Description | Default |
|---|---|---|---|
| then | No | desktop | |
| steps | Yes | ||
| stop_on_change | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden, and it does so thoroughly. It discloses ordering guarantees, failure semantics (a failing step stops the run and reports which one), the need for window= for reliable typing, opt-in stop_on_change behavior including its false-positive caveat, and the `then` observation options. This goes well beyond a basic statement of what the tool does.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is dense but every sentence earns its place: the core purpose is front-loaded, the example makes the schema concrete, and each caveat (error stopping, window focusing, stop_on_change tradeoffs, then-coverage) is necessary for correct use. It is long only because the tool is genuinely nuanced.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a composite tool with meaningful behavioral nuance, yet the description covers all three parameters, the valid step ops, failure/cancellation semantics, observation modes, and common pitfalls. Since an output schema exists, not describing return values in prose is acceptable. Nothing essential for an agent to decide whether and how to call it is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, and it fully does. It defines the steps array with an exact JSON shape and example, explains the op enum, clarifies `window=` semantics, and explicitly names `stop_on_change` and `then` with their defaults and behavior. Every parameter is given meaning beyond its schema title.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific verb-resource pair: 'Run an ordered list of actions in ONE call', and distinguishes sequence from single-action siblings by emphasizing that multiple steps can be batched into one round-trip. It also names the supported step types (pointer, keyboard, click_ui, hypr, wait_for), making its role unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: use sequence when you need an ordered micro-sequence of actions like click/type/enter, and it explains the round-trip benefit over issuing several calls. It does not explicitly state when not to use it or name alternatives by name, but the 'vs several calls' framing effectively implies the alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
uiA
Read a window's accessibility tree (AT-SPI) and return its elements
with GLOBAL click points, so you can target a control by NAME with no
screenshot and no pixel guessing. window is an address from desktop
(default: the focused window). name filters to elements whose
accessible name contains it (case-insensitive); actionable (default)
keeps only interactive roles (buttons, entries, menu items, ...).
Returns [{role, name, x, y, clickable}] where x,y is the click point:
focus the window, then click it with pointer (the window must be
visible to receive the click), or do both in one call with click_ui.
Controls that carry a CURRENT VALUE also
report it: value (text typed into an entry, or a slider/spinner
number), percent for a slider's position, checked for a box or
toggle. Password fields never report contents, and many dropdowns
expose no value at all, so read the screen with screenshot when a
rendered value matters. Not every app exposes a tree (terminals, and
Electron/Chrome without --force-renderer-accessibility, expose little
or nothing); when it does not, fall back to screenshot + zoom.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | ||
| window | No | ||
| actionable | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses important behaviors: the tree may not be available for certain apps, password fields never report contents, dropdowns may lack value, and click points are global requiring window focus. It also mentions that the tool returns elements with roles, names, and click points. The description does not contradict annotations (none provided).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is detailed and well-structured, front-loading the main purpose. Each sentence adds value, though it is slightly long. There is no redundancy, and the technical details are clearly presented. It could be slightly more concise, but overall it is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (accessibility tree with varying behavior), the description is remarkably complete. It explains limitations, fallback strategies, return format, and exceptions. It covers all aspects needed for correct invocation, including implicit output schema description. No output schema is provided, but the description compensates with clear return structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It thoroughly explains all three parameters: 'window' (address from desktop, default focused), 'name' (filters by accessible name, case-insensitive), and 'actionable' (default true, keeps interactive roles). It also describes the return structure, adding significant value beyond the input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it reads the accessibility tree and returns elements with click points, distinguishing it from sibling tools like screenshot (visual fallback) and pointer (clicking). It specifies the action 'Read a window's accessibility tree' and the resource 'elements...with GLOBAL click points'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to use the tool (to target a control by name without screenshot or pixel guessing), and when to fall back (when the tree is not exposed, like in terminals or Electron/Chrome). It also mentions limitations (password fields, dropdowns) and suggests alternatives (click_ui, screenshot). However, it does not explicitly state when NOT to use this tool versus specific siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
use_bindA
Run one of the user's own Hyprland keybinds by its combo (from the
binds tool), e.g. 'SUPER+F'. This executes the bound action directly
(the only reliable way: synthetic keypresses do not trigger compositor
binds). Use it to drive the owner's configured workflows: launchers,
layout shortcuts, scratchpads. then ('desktop'|'screenshot'|'ui'|'none')
appends the result to this call (handy after a launcher bind).
| Name | Required | Description | Default |
|---|---|---|---|
| then | No | none | |
| combo | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that the tool executes the bound action directly (not via synthetic keypresses) and that the `then` parameter appends the result. It adds meaningful behavioral context, though it does not cover failure modes, permissions, or side-effect boundaries beyond what the schema implies.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences cover purpose, rationale, use cases, and parameter semantics without filler. The core action is front-loaded, and every sentence contributes to correct usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (2 parameters, 1 required) and has an output schema, so return values are already covered. The description addresses what, why, when, and both parameters. Minor gaps include an explicit prerequisite that the combo must already exist in the user's binds (though implied by 'from the binds tool') and error behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It defines `combo` as a keybind combo sourced from the `binds` tool with an example, and lists the valid `then` values ('desktop'|'screenshot'|'ui'|'none') with a note that it appends the result. This is valuable, though the precise effect of each `then` value is not elaborated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Run'), resource ('user's own Hyprland keybinds'), and mechanism ('by its combo'), with a concrete example ('SUPER+F'). It distinguishes itself from synthetic keypresses and references the `binds` sibling, making the tool's niche unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly states when to use the tool ('to drive the owner's configured workflows') and why it should be preferred ('the only reliable way') over synthetic keypresses, which are called out as non-functional. This provides clear when/when-not guidance without needing to enumerate sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
wait_forA
Block until a desktop event happens (real compositor events, not
polling). event: 'window_open' | 'window_close' | 'workspace' |
'title_change' | 'layer_open' | 'layer_close' (layer-shell surfaces:
launchers, notification popups; match on the namespace, e.g. 'wofi') |
'urgent' (a window demands attention) | 'screencast' (screen sharing
started/stopped). match: optional case-insensitive substring filter
over the event's fields (class/title/workspace name/address/namespace).
timeout_s 1-60, default 10. Returns the event payload, or a timeout
note; a filtered wait whose condition ALREADY holds (the window is
already gone, the workspace already active, the layer already mapped)
returns instantly with already: true instead of missing an event
that fired before it could subscribe. Use it after actions with
delayed effects: app startups, page loads that change a window title,
a launcher bind that pops a layer.
| Name | Required | Description | Default |
|---|---|---|---|
| event | Yes | ||
| match | No | ||
| timeout_s | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the burden of behavioral disclosure. It details that it blocks on real compositor events (not polling), returns the event payload or a timeout note, and that a filtered wait whose condition already holds returns instantly with `already: true`. It also explains that matching is case-insensitive substring filtering. This is thorough behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but every sentence adds value. It is front-loaded with the core purpose ('Block until a desktop event happens, not polling'), then details event types, matching, timeout, return behavior, and usage examples. There is no fluff; each clause conveys essential information. The structure is logical and efficient for the complexity of the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the tool (multiple event types, matching logic, timeout semantics, edge case for conditions already holding), the description covers all aspects necessary for an agent to use it correctly. It includes examples of when to use it and explains the behavior for every scenario. The presence of an output schema means return-value details are not required in the description, but it still mentions what the tool returns. This is fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It does so comprehensively: the `event` parameter is fully enumerated with descriptions for each event type (e.g., 'layer-shell surfaces: launchers, notification popups; match on the namespace, e.g. wofi'), `match` is explained as an optional case-insensitive substring filter over the event's fields, and `timeout_s` is given a range (1-60) and default (10). This fully compensates for the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the primary purpose: 'Block until a desktop event happens', and then enumerates the specific event types supported. It distinguishes this tool from its siblings (e.g., launch, use_bind, screenshot) by focusing on waiting for events rather than performing actions. The agent can immediately understand what the tool does and how it differs from the other tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when to use the tool: 'Use it after actions with delayed effects: app startups, page loads that change a window title, a launcher bind that pops a layer.' It also explains the behavior when the condition already holds, which helps the agent anticipate edge cases. This is clear and actionable usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
zoomA
Native-resolution re-capture around a point: the precision step of
the coarse-to-fine loop. Screenshot first, estimate the target's global
x,y, zoom there, re-estimate on the zoomed image (scale ~1.0, so
global = geometry[:2] + image_pixel), then click. size "WxH" in
logical pixels (default 480x360) is clamped to the screen; window
(an address from desktop) clamps to that window instead. The metadata
echoes the requested point back as point; stable=true waits for
the frame to settle first; lossless=true returns PNG.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| size | No | ||
| stable | No | ||
| window | No | ||
| lossless | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
In the absence of annotations, the description does a good job disclosing key behaviors: size clamping, window constraint, stable waiting, lossless PNG, and coordinate transformation. However, it omits potential side effects or requirements like permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is somewhat dense and technical, mixing procedural steps with parameter details. It is front-loaded with purpose but could be more structured (e.g., bullet points) for clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the availability of an output schema, the description covers workflow and parameter behavior well. It lacks error handling or prerequisites but is adequate for a visual interaction tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema coverage, the description adds significant meaning to all parameters: explains 'size' format and clamping, 'window' as address, 'stable' for frame settling, and 'lossless' for format. x/y are implied but not explicitly described, leaving minor ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's role as 'Native-resolution re-capture around a point: the precision step of the coarse-to-fine loop.' It explains the workflow (screenshot, estimate, zoom, re-estimate, click) and distinguishes from sibling tools like 'screenshot' by focusing on refinement.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage within a coarse-to-fine loop but does not explicitly state when to use this tool versus alternatives like 'screenshot' or 'desktop'. No 'when-not-to-use' guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
14 tool updates
v0.11.0- First observed
binds - First observed
click_ui - First observed
desktop - First observed
hypr - First observed
keyboard - First observed
launch - First observed
marks - First observed
pointer - First observed
screenshot - First observed
sequence - First observed
ui - First observed
use_bind - First observed
wait_for - First observed
zoom
TDQS
Scored across 14 tools
Each tool targets a distinct capability: hypr for compositor window/workspace ops, desktop for state snapshots, screenshot/zoom for capture, ui/marks/click_ui for accessibility-driven interaction, pointer/keyboard for raw input, binds/use_bind for user-defined shortcuts, wait_for for events, launch for starting apps, and sequence for batching. Overlaps are minimal and clearly delineated (e.g., click_ui vs pointer).
Names are a mix of single-word nouns (desktop, ui, marks, binds, pointer, keyboard, sequence) and verbs (launch, zoom, screenshot), plus compound verb-noun forms (wait_for, click_ui, use_bind). The inconsistency is not chaotic but lacks a uniform pattern like verb_noun throughout.
14 tools is well-scoped for a desktop automation server covering input, capture, state, events, launching, and UI interaction. Each tool has a clear role with no redundancy, and the count is neither thin nor bloated.
The surface covers the full lifecycle of desktop automation: observing (desktop, screenshot, zoom), interacting (pointer, keyboard, click_ui, hypr), waiting for changes (wait_for), launching (launch), and executing user workflows (binds, use_bind). No obvious gaps for the stated purpose of controlling Hyprland desktops.
Maintenance
Related MCP Connectors
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to interact with yo…
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Melaya is a remote MCP server. It gives an assistant hands on your own Android phone and browser: it reads the screen through the accessibility tree, then taps, types and navigates inside the apps and sites you allow-list, with no per-app API. It also builds, schedules and runs agent pipelines across 6k+ connected tools. OAuth 2.1, nothing to install.
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceEnables LLMs to capture screenshots, inspect windows, and interact with the Hyprland desktop environment through mouse clicks and keyboard input.1-
- AlicenseAqualityAmaintenanceAn MCP server for Hyprland that enables AI agents to control workspaces, windows, mouse, keyboard, and take screenshots on a Wayland desktop.14249 PyPI24MIT
- FlicenseAqualityAmaintenanceCross-platform desktop automation MCP server that lets AI agents capture screenshots, run OCR with UI-element classification, control mouse/keyboard, and launch programs on Linux, macOS, and Windows.201-
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to automate a Linux desktop natively on Wayland, with 68 tools for screen capture, input simulation, browser automation, OCR, accessibility, media, audio, clipboard, notifications, filesystem, system, memory, and network operations through the MCP protocol.MIT