Click on "Install Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@Hands take a screenshot of my desktop"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
A macOS computer-use MCP server for autonomous AI agents.
Give an LLM eyes, a mouse, and a keyboard — with a real security model in between.
Quick Start • Tools • Security Model • Plugins • Docs
Most "computer use" demos give a model raw pixel coordinates and hope for the best. Hands is built differently: every action goes through a typed 7-phase dispatch pipeline — validate → rate-limit → authorize → lock → execute-with-retry → observe → audit — so that giving an agent control of your desktop doesn't mean giving up any of your say in what it's allowed to do.
import anyio
from hands import Container, HandsConfig
config = HandsConfig()
config.driver = "fake" # no macOS needed — great for a first look
container = Container.build(config)
async def main():
shot = await container.dispatcher.dispatch("screenshot", {})
print(shot["ok"], shot["bounds_pt"])
anyio.run(main)A real policy engine, not a rubber stamp. Rule-based profiles
(strict / default / trusted), an app deny-list (Passwords,
1Password, System Settings — blocked out of the box), secret-pattern
detection on typed text, per-tool confirmation, and a sliding-window rate
limiter. See Security Model.
Tamper-evident audit log. Every action is appended to a SHA-256
hash-chained JSONL log; deleting or editing any line downstream breaks the
chain, and hands audit verify proves it.
Redaction by construction. Clipboard content and typed text never enter application state, logs, audit records, or metrics — only their length and a SHA-256 hash ever do.
A fake driver, not just mocks. FakeDriver is a full in-memory virtual
desktop (windows, apps, clipboard, AX tree, OCR boxes) that every tool runs
against identically to the real macOS driver. The entire test suite — 215
tests — runs on Linux CI with zero macOS hardware, and it's the fastest way
to try Hands out or build a plugin without touching your real screen.
Perception, not just action. OCR-grounded text search (find_text),
an accessibility-tree fallback (get_ui_tree), and a verify/wait
condition language so an agent can confirm an action worked instead of
flying blind.
A kill switch that actually works. touch ~/.hands/KILL halts the
server immediately — checked before every single dispatch, including
mid-way through a batched execute_sequence.
Extensible without forking. Third-party plugins register tools through
the exact same ToolSpec machinery as the 22 built-ins, loaded via Python
entry points; a plugin that raises during setup is logged and skipped, not
a server crash. (Plugins run in-process with access to the same services —
this is crash isolation, not a security sandbox; don't load untrusted
plugin code any more than you would an untrusted import.)
Related MCP server: macos-control-mcp
macOS 13+ (Ventura or later) for real desktop control
Python 3.12+
Screen Recording and Accessibility permissions (System Settings → Privacy
& Security) — hands permissions tells you exactly what's missing
You can install and explore the library on any OS using the fake driver
(HANDS_DRIVER=fake) — only real desktop control requires macOS.
pip install hands-mcpWith the macOS driver and Vision OCR (needed for real desktop control):
pip install "hands-mcp[macos]"The import name is still hands regardless of the install name above:
import hands
print(hands.__version__)For development:
git clone https://github.com/yuvitbatra/hands-mcp.git
cd hands-mcp
uv sync --group dev
uv run pytest -qAdd to your MCP client configuration (e.g. Claude Desktop's
claude_desktop_config.json):
{
"mcpServers": {
"hands": {
"command": "hands",
"args": ["serve"]
}
}
}Or run it directly over stdio:
hands serveimport anyio
from hands import Container, HandsConfig
# The fake driver is a full in-memory virtual desktop — no macOS needed.
config = HandsConfig()
config.driver = "fake"
container = Container.build(config)
async def main():
res = await container.dispatcher.dispatch("screenshot", {})
print(res["ok"]) # True
anyio.run(main)# Or run the full MCP server over stdio from your own process:
import anyio
from hands import run_server, load_config
anyio.run(run_server, load_config())hands doctor # resolved config, driver, displays, registered tools
hands doctor --metrics # ...plus a metrics snapshot (counters + latency histograms)
hands permissions # TCC grant status (Screen Recording, Accessibility) + fix links
hands audit verify # verify the audit log's hash chain end to endFull argument/return reference: docs/TOOLS.md.
Tool | Policy | Description |
| read | Capture the screen (or a region); cached by default |
| read | Session state: cursor, displays, action history, dirty flag |
| read | OCR search returning bounding boxes for matching text |
| read | Accessibility tree for an app (roles, labels, frames) |
| read | Poll until a condition is met: text present, window state, duration |
| read | Check an expected outcome after acting; returns evidence |
| act | Move the cursor |
| act | Click at a point (single, double, or triple) |
| act | Click-drag along a path |
| act | Scroll at a point |
| act | Type text into the focused element (refused during secure input) |
| act | Press a key or chord (e.g. |
| sensitive | Read the current clipboard (text or image) |
| act | Set the clipboard to text or a PNG image |
| act | Paste via clipboard + Cmd+V, restoring the clipboard after |
| read | List windows, optionally filtered by app |
| act | Bring a window to front (stale refs re-resolve by fuzzy title) |
| act → sensitive on | Move, resize, minimize, maximize, or close a window |
| act | Launch an app by bundle ID or name, waiting for its window |
| act → sensitive on | Quit an app gracefully, or force-terminate |
| read | List running apps and the frontmost one |
| act | Batch up to 20 pre-decided acting steps in one round trip, gated by guard conditions |
An agent that can move your mouse and type on your behalf is a genuinely different risk profile from a chatbot, so the policy layer isn't an afterthought — it's a fixed phase of every dispatch, before the tool ever runs:
validate args → rate limit → authorize (PermissionEngine) → acquire action lock
→ execute with retry → record state/screen-dirty → audit + metricsProfiles (HANDS_SECURITY__PROFILE, default default):
Profile |
|
|
| requires confirmation | requires confirmation |
| allowed | requires confirmation |
| allowed | allowed |
Evaluation order for every action: (1) is the frontmost app on the
deny-list? → denied outright, even under trusted. (2) does a configured
rule match? → first match wins. (3) does typed/pasted text match a
secret_patterns regex? → confirmation required regardless of profile. (4)
fall through to the profile's class default above.
App deny-list (HANDS_SECURITY__DENY_APPS, on by default): blocks
acting tools — reads still work — against com.apple.systempreferences*,
com.apple.Passwords*, com.apple.keychainaccess, 1Password (both bundle
id families). This applies uniformly whether the tool is called directly or
from inside execute_sequence.
Rate limiting (HANDS_SECURITY__MAX_ACTIONS_PER_S, default 10.0): a
1-second sliding window over acting tools; bursts past the limit are denied,
not queued.
Secure input refusal: keyboard_type and clipboard_get are refused
outright while macOS reports secure text entry active (a password field has
focus) — independent of profile.
Confirmation hooks: under dialog mode (default, macOS only) a
denied-by-default sensitive action pops a real confirmation dialog via
osascript; under deny mode (or off-macOS) it's auto-denied. Plug in your
own hook by subclassing the PermissionEngine's confirm callback if you need
a different UX (e.g. routing through your agent framework's own approval
flow).
Redaction invariant: clipboard content and typed text are used only
in-memory for policy matching (e.g. secret-pattern checks) — they never
reach state, the audit log, or metrics in raw form. Only length and a
SHA-256 hash ever leave those boundaries.
Kill switch: touch ~/.hands/KILL halts the server immediately —
checked at the top of every dispatch, including mid-sequence inside
execute_sequence. Remove the file to resume.
Audit log: every action is appended to ~/.hands/audit.jsonl as
{"event", "prev_hash", "hash"}, where hash = sha256(prev_hash + canonical_json(event)). Tampering with or deleting any line downstream
breaks the chain — verify with hands audit verify.
All settings can be overridden via HANDS_* environment variables (nested
fields use __, e.g. HANDS_SECURITY__PROFILE). Full schema:
src/hands/config.py.
Variable | Default | Description |
|
|
|
|
|
|
| see above | JSON list of bundle-id globs blocked for acting tools |
|
| JSON list of regexes that force confirmation on typed/pasted text |
|
| Sliding-window rate limit for acting tools |
|
|
|
| (all allowed) | JSON list of allowed plugin entry-point names |
|
| Max screenshot dimension |
|
| JPEG quality (0–100) |
|
| Delay between mouse down and up |
|
| Characters per typing chunk |
|
| Depth-first-traversal node cap for |
Hands supports external plugins that register custom tools through the exact
same ToolSpec machinery as the built-ins — no side door around the
dispatcher, so plugin tools get the same policy, retry, and audit treatment.
from hands.plugins.api import HandsPlugin, PluginContext
class MyPlugin:
name = "my-plugin"
version = "1.0.0"
def setup(self, ctx: PluginContext) -> None:
from pydantic import BaseModel
from hands.registry import ToolSpec
from hands.retry import RetryPolicy
class Args(BaseModel, extra="forbid"):
query: str
async def my_tool(args: Args, ctx_) -> dict:
return {"result": f"searched for {args.query}"}
ctx.registry.register(ToolSpec(
"my_search", "Custom search tool",
Args, my_tool, "read", RetryPolicy.read(), idempotent=True))
def teardown(self) -> None:
passRegister it via the hands.plugins entry-point group in your own package's
pyproject.toml:
[project.entry-points."hands.plugins"]
my-plugin = "my_package.plugin:MyPlugin"Plugins load at server startup. A plugin that raises in setup is logged
and skipped — it never takes the server down. Restrict which plugins may
load with an allowlist:
HANDS_SECURITY__PLUGIN_ALLOWLIST='["my-plugin"]' hands serveSee docs/plugins.md for the full plugin author's guide,
and src/hands/plugins/api.py for the stable
import surface (semver-guarded — additive-only within a major version).
uv run pytest -q # 215 tests, any OS, ~2s
HANDS_CONTRACT_MACOS=1 uv run pytest tests/contract -q # real macOS driver (needs TCC grants)
HANDS_E2E_MACOS=1 uv run pytest tests/e2e -q # full stack vs. a real Tk fixture app
uv run pytest tests/perf -m perf --benchmark-only -q # latency budgets
uv run pytest tests/stress -m stress -q # 10k-action soak + concurrencyThe contract/e2e suites are opt-in because they genuinely move your mouse
and open/close real apps — don't run them unattended. See
CONTRIBUTING.md for the full development workflow.
types, errors, config, retry
↓
driver/base (Protocol) ← fake.py (tests/CI) + macos.py (real hardware)
↓
services/ (screenshot, ocr, mouse, keyboard, clipboard, windows, apps,
waiter, verification)
↓
state, permissions (PermissionEngine), audit (hash-chained), metrics, registry
↓
dispatcher (7-phase pipeline: validate → rate-limit → authorize → lock →
execute-with-retry → observe → audit)
↓
tools/ (22 MCP tools)
↓
container → server (MCP stdio transport) → cliFull design rationale: docs/DESIGN.md.
Contributions welcome — see CONTRIBUTING.md for the
development setup, test-driven workflow, and what a good PR looks like.
Quit an app gracefully; force=true force-terminates (sensitive, needs confirmation under the default policy).
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | ||
| force | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It usefully notes that force=true force-terminates and is 'sensitive, needs confirmation under the default policy', which adds meaningful context beyond the schema. The graceful default behavior is implied but not deeply detailed, yet sufficient for most use cases.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the main action and includes the critical force nuance. Every word adds value; there is no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two parameters and no output schema, the description is largely complete. It covers the core behavior and the sensitive force option, and the overall scope is clear. The only minor gap is the lack of specification for the app parameter format, but this is not critical for basic selection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero descriptions (0% coverage), so the description must compensate. It explicitly explains the force parameter ('force=true force-terminates'), adding semantic value. However, the app parameter is only referred to as 'an app' without specifying expected format or identifier, leaving some ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Quit an app gracefully' with a specific resource ('app'), and distinguishes itself from sibling tools like app_open and app_list by focusing on termination. It also clarifies the force mode, making the tool's purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use the tool (quitting apps) and the optional force behavior. While it does not explicitly contrast with alternatives like window_manage, the sibling context and wording make the intended use evident. There are no exclusions or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List running apps and the frontmost one.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It implies a read-only operation but does not detail behavior like whether the list is sorted, whether the frontmost app is included in the list, or any platform-specific quirks. Still, the simple action is clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that efficiently communicates the tool's purpose with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given zero parameters and no output schema, the description covers the primary purpose but lacks detail on the exact return structure (e.g., app names, identifiers) or any edge cases. It is adequate for a simple listing tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has zero parameters, so the description is not required to explain parameters. Baseline 4 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (list) and the resource (running apps, frontmost one), distinguishing it from window_list by focusing on applications rather than windows.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this vs. alternatives like window_list or get_state. The need is implied by the nature of the tool, but no exclusions or sibling comparisons are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Launch an app by bundle id (preferred) or name; activates it if already running. wait_for_window waits for its first window.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | ||
| timeout_ms | No | ||
| wait_for_window | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that the tool activates an already running app and that wait_for_window makes it wait for the first window. This goes beyond mere launch semantics, but it does not explain error handling or timeout behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no filler. It is front-loaded with the core action and efficiently incorporates behavioral context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, but the description does not cover all aspects an agent might need. It omits timeout_ms semantics and potential failure modes, and it does not clarify what happens when wait_for_window is false. Given no output schema, the description is functional but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaning to the 'app' parameter (bundle id preferred, or name) and to 'wait_for_window' (waits for first window), but it completely omits 'timeout_ms'. Since schema descriptions are absent (0% coverage), the description only partially compensates for the gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Launch an app by bundle id (preferred) or name.' It uses a specific verb and resource, and the activation behavior distinguishes it from siblings like app_close or app_list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides some usage guidance by preferring bundle id over name, but it does not explicitly say when to use this tool versus alternatives like window_focus or app_list. It also lacks exclusions or when-not-to-use scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Read the clipboard (sensitive: may require user confirmation). Refused while a password field has focus.
| Name | Required | Description | Default |
|---|---|---|---|
| format | No | any |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of disclosure. It openly flags the operation as sensitive, warns about possible user confirmation, and explains a specific scenario where the tool will be refused. This is strong behavioral transparency, though it omits details about return behavior on denial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the core purpose, and includes only essential caveats. Every word earns its place, with no fluff or redundant restatements of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with one optional parameter and no output schema, the description covers the primary action and key constraints. However, the complete omission of the 'format' parameter leaves a significant gap in understanding how the tool behaves with different inputs, so it is merely adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It fails to mention the 'format' parameter entirely, leaving its meaning and impact undisclosed beyond the bare schema enum. The agent cannot infer which format to request or the resulting output behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Read' and resource 'clipboard', distinguishing it from sibling clipboard_set and clipboard_paste. It also adds context about sensitivity and a specific refusal condition, leaving no ambiguity about the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context about when the tool may require confirmation and explicitly states when it will be refused (password field focus). However, it does not name alternatives or explicitly state 'use this when...', so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Paste text into the focused app via clipboard + Cmd+V, then restore the previous clipboard. Preferred over keyboard_type for long text.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| restore | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full transparency burden. It discloses a key side effect: the clipboard is modified and then restored, which is critical for the agent to know. It doesn't cover failure modes or the restore=false scenario, but the main behavior is clearly revealed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no redundant words. The first sentence states purpose and mechanism; the second gives usage guidance. Perfectly succinct.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no output schema and no annotations, the description covers purpose, mechanism, a usage guideline, and the crucial clipboard restore behavior. It lacks explicit mention of the restore parameter's optionality and potential failure conditions, but these are minor given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% property descriptions, so the description must compensate. It indirectly explains the 'text' parameter and the 'restore' parameter ('then restore the previous clipboard'), but it doesn't state that restore can be set to false or what that changes. This is adequate but not fully comprehensive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific action ('Paste text into the focused app') with a precise mechanism ('via clipboard + Cmd+V') and distinguishes itself from sibling 'keyboard_type' by noting it is preferred for long text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says 'Preferred over keyboard_type for long text,' giving a clear alternative and a context for use. It does not mention when not to use it, but the guidance is sufficient for choosing between the two tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Set the clipboard to text or a base64 PNG.
| Name | Required | Description | Default |
|---|---|---|---|
| text | No | ||
| image_b64 | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full transparency burden. It states the basic action but omits critical behavioral details: what happens if both 'text' and 'image_b64' are provided, whether they are mutually exclusive, size limits (though schema has maxLength for text), and error behavior. The absence of these disclosures leaves ambiguity for the agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, focused sentence that communicates the core function without any wasted words. It is appropriately sized for a simple tool and immediately front-loads the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple clipboard setter, the description covers the essential function, but gaps remain: no mention of mutual exclusivity or precedence of parameters, no handling of empty input, and no note about whether the operation is synchronous or has side effects. These missing edge cases make it adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers 0% of parameter descriptions, so the description must compensate. It successfully maps 'text' to text content and 'image_b64' to a base64 PNG, and the word 'or' implies mutual exclusivity. However, it doesn't explain constraints (e.g., max length, encoding requirements) or clarify what happens if neither parameter is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Set') and resource ('the clipboard'), clearly distinguishing it from siblings like clipboard_get and clipboard_paste. It also enumerates the two supported content types (text, base64 PNG), making the tool's scope unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (any time you want to set clipboard content) but offers no explicit guidance on alternatives, exclusions, or when to choose text versus image. It relies on the obviousness of the operation rather than providing directional advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Run up to 20 PRE-DECIDED acting steps in one call (click, type, press...), each optionally gated by a guard condition (same schema as wait). A failed guard halts the sequence and returns evidence. Use only when you already know every step; this is a macro, not a planner.
| Name | Required | Description | Default |
|---|---|---|---|
| steps | Yes | ||
| stop_on_failure | No | ||
| screenshot_after | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses key behaviors: guard conditions can gate steps, failed guards halt the sequence, and evidence is returned. It also emphasizes the 'pre-decided' nature and non-planner role. However, it does not explain behavior around stop_on_failure for ordinary step failures, nor what 'evidence' includes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences that are dense with meaning. It front-loads the core function, then adds important usage context. Every word contributes value with no redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (nested steps, multiple options, no output schema or annotations), the description covers the main purpose and guard behavior but omits crucial details about parameter structure, failure handling for regular step failures, and the nature of returned evidence. It is minimally viable but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It only clarifies the guard parameter and implicitly the steps limit, but does not explain the structure of each step (tool, args), nor the purpose of stop_on_failure or screenshot_after. This leaves significant gaps for a complex nested schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool runs up to 20 pre-decided acting steps in one call, naming the action (run), resource (acting steps), and scope (up to 20). It distinguishes itself from sibling low-level tools by framing itself as a macro, not a planner.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is given: 'Use only when you already know every step; this is a macro, not a planner.' This gives a clear condition for when to use the tool, but it does not explicitly name alternatives (e.g., individual action tools). The reference to `wait` for guard schema adds context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
OCR the screen (or a region, in points) and return boxes matching text. Each match has a center you can pass directly to mouse_click. Re-observe rather than act when confidence < 0.5. By default this matches fuzzily (case-insensitive substring either direction, or a similarity ratio >= 0.8); pass fuzzy=false for exact substring matching. This differs from wait's text_present condition and verify's text_present expectation, which always require an exact substring match.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| fuzzy | No | ||
| region | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral disclosure. It details return value characteristics (boxes with a center), fuzzy matching default (case-insensitive substring or similarity ratio >= 0.8), the fuzzy parameter to disable it, region support in points, and the confidence threshold guidance. This is far beyond a basic purpose statement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each packing essential information without waste: purpose, usage/confidence advice, and matching modes with sibling comparison. The structure is front-loaded with the core function and flows logically. No redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no annotations and no output schema, the description is sufficiently complete. It covers purpose, usage versus alternatives, parameter meanings, behavioral nuances, and actionable guidance. An agent can correctly select and invoke this tool based on the description alone.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has zero descriptions, but the description compensates by explaining the text parameter (string to match), fuzzy parameter (default true, semantics of matching), and region parameter (optional, in points). It adds meaningful context that the raw schema lacks, especially the fuzzy behavior and region coordinate system.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence clearly identifies the tool's function: 'OCR the screen (or a region, in points) and return boxes matching `text`.' It uses a specific verb (OCR) and resource (screen/region), and distinguishes itself from sibling tools wait and verify by explicitly contrasting the matching semantics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: it contrasts with wait and verify which require exact substring matches, indicating when the fuzzy capability is useful. It also offers a confidence-based tip ('Re-observe rather than act when confidence < 0.5') and mentions passing the returned center directly to mouse_click, which suggests a common downstream action.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Re-orientation: cursor position, displays, last screenshot metadata, kill-switch status, and recent action history.
| Name | Required | Description | Default |
|---|---|---|---|
| include_history | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description must carry the burden of behavioral disclosure. It does not explicitly state that the tool is read-only or side-effect-free, but the wording 'Re-orientation' and the act of listing state items strongly imply a non-destructive observation. It also fails to mention aspects like whether include_history affects output or how metadata is returned, leaving some behavioral ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence, front-loading the purpose with 'Re-orientation' and listing the key state components without redundancy or filler. Every word contributes meaningful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has low complexity (one optional parameter, no output schema), and the description lists the main state categories, which provides a basic understanding. However, it lacks details about the form of the outputs (e.g., whether cursor position includes coordinates, what 'displays' means, how history is structured) and does not explain the include_history parameter, leaving the description incomplete for an agent to anticipate exact return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has one optional parameter, include_history, with type, default, and range but no semantic explanation. The description mentions 'recent action history' as part of the state, but does not link it to the parameter or clarify how the integer value controls history inclusion (e.g., whether 0 means none). With 0% schema description coverage, the description should compensate, but it does not adequately explain parameter behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a state/orientation snapshot, enumerating specific contents: cursor position, displays, last screenshot metadata, kill-switch status, and recent action history. It distinguishes itself from sibling action tools by listing these unique state components, though it lacks an explicit verb like 'retrieve' or 'get', relying on the tool name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The term 'Re-orientation' implies using this tool when an agent needs to determine current state, but it does not explicitly state when to use versus alternatives or provide exclusions. There are no alternative tool names or scenarios mentioned, leaving the usage context somewhat implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Accessibility tree for an app (frontmost if omitted): roles, titles, values, clickable regions in points. Ground truth where apps expose it; use find_text where they don't.
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | ||
| max_depth | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the output content, coordinate units, and the reliability caveat ('ground truth where apps expose it'). It does not explicitly state it is read-only or mention permission requirements, but for a read-like tool this is strong.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core purpose and output format, with no unnecessary detail. Every clause adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description covers purpose, app parameter semantics, output contents, and a fallback alternative. It does not document max_depth or return formatting details, but these are largely inferable from the schema and tool name, making it complete enough for selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description clarifies the app parameter by stating it defaults to the frontmost app, matching the schema default. However, max_depth is never mentioned, and schema description coverage is 0%, so the description only partially compensates for the lack of parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states what the tool does: returns an accessibility tree for an app with roles, titles, values, and clickable regions in points. It also distinguishes itself from sibling find_text by noting the appropriate tool for cases where accessibility data is not exposed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly says to use find_text when apps don't expose the accessibility tree, providing a clear alternative and exclusion condition. It also notes that the app parameter is optional and defaults to the frontmost app.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Type text into the focused element using layout-independent unicode injection. Click the target field first.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| chunk_delay_ms | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the injection method and the need for focus, but does not explain chunk_delay_ms behavior, handling of special characters, or error behavior. This is partial transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, directly addresses the core action, and front-loads the key information without any filler. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description covers the basic use case but omits details like chunking behavior, the effect of the delay parameter, and failure modes. It is minimally sufficient but has clear gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description does not explain parameters. 'text' is obvious from context, but 'chunk_delay_ms' is left unexplained. The description only indirectly maps to the text parameter, failing to compensate for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Type text into the focused element' with a specific method (layout-independent unicode injection). It distinguishes itself from sibling tools like key_press and clipboard_paste by its scope and mechanism.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides a clear prerequisite: 'Click the target field first.' This gives concrete guidance for when to use the tool, but it does not explicitly contrast with alternatives or mention scenarios where it should not be used, so it is not a full 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Press a key or shortcut chord, e.g. 'Return', 'cmd+s', 'cmd+shift+p', 'F5'. Use keyboard_type for regular text.
| Name | Required | Description | Default |
|---|---|---|---|
| chord | Yes | ||
| repeat | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It clarifies that this is for keys/shortcuts and not text, but does not disclose other behaviors like synchronous execution, repeat semantics, or behavior on invalid chords. It's adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one concise sentence with examples and a pointer to an alternative. It is front-loaded with the action and resource, and every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two parameters and no output schema, the description is largely complete. It covers the primary use case and provides a clear alternative. The main gap is the lack of detail on 'repeat' behavior, but the overall tool is simple enough that this is a minor omission.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It provides useful format examples for the 'chord' parameter (e.g., 'cmd+s', 'F5'), but does not explain the 'repeat' parameter at all. The schema gives defaults and bounds, but the agent lacks guidance on what 'repeat' actually does.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Press a key or shortcut chord' with specific examples like 'Return', 'cmd+s', and 'F5'. It distinguishes from sibling keyboard_type by explicitly noting that tool is for regular text, making the purpose and scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use keyboard_type for regular text,' which tells the agent when to use this tool versus the alternative. It also implies key_press is for keys and shortcuts, providing clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Click at (x, y), or at the current cursor if omitted. Coordinates are logical points, origin top-left of the main display. Compute them from the screenshot tool's bounds_pt and px_per_pt metadata. After clicking, take a screenshot to verify the result.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| clamp | No | ||
| count | No | ||
| button | No | left | |
| modifiers | No | ||
| require_fresh_screenshot | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral disclosure burden. It discloses the coordinate system (logical points, origin top-left), fallback to current cursor when coordinates are omitted, and the need to derive coordinates from screenshot metadata. It also advises verification. However, it omits details about modifier handling, clamp behavior, button semantics, and require_fresh_screenshot.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the primary action, followed by coordinate system clarification and verification advice. Every sentence adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 7 parameters, no annotations, and no output schema, the description covers the core click action and coordinate system but leaves many parameters and return behavior unexplained. It provides enough for basic usage but not full contextual completeness given the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains x and y (coordinate meaning and origin) and that omitting them clicks at the current cursor, but provides no explanation for clamp, count, button, modifiers, or require_fresh_screenshot. With 7 parameters and 5 unexplained, parameter semantics is weak.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Click at (x, y), or at the current cursor if omitted') and identifies the resource (mouse). It distinguishes from sibling tools like mouse_move and mouse_drag by specifying click behavior and coordinate semantics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides practical usage context by explaining coordinate computation from screenshot metadata and recommending a post-click screenshot verification. However, it does not explicitly contrast with alternatives like mouse_move or mouse_drag, nor state when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Press, drag along path, release. Coordinates are logical points, origin top-left of the main display. Compute them from the screenshot tool's bounds_pt and px_per_pt metadata.
| Name | Required | Description | Default |
|---|---|---|---|
| path | Yes | ||
| button | No | left | |
| duration_ms | No | ||
| require_fresh_screenshot | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It discloses the press-drag-release sequence and the coordinate system origin (top-left of main display). It does not mention potential side effects, button behavior during drag, or consequences of invalid paths, leaving some behavioral aspects opaque.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise: two sentences. The first sentence states the action and the second provides coordinate computation context. There is no redundant information or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description does not need to specify return values. It covers the core operation and coordinate system, but leaves optional parameter behaviors (button, duration, fresh screenshot requirement) unexplained. For a tool of moderate complexity, this is a notable gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description adds meaningful semantics for the 'path' parameter, explaining that coordinates are logical points and how to compute them from screenshot metadata. However, it provides no explanation for 'button', 'duration_ms', or 'require_fresh_screenshot', which is notable given the schema description coverage is 0%.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the action: 'Press, drag along path, release.' This clearly indicates a mouse drag operation, distinguishing it from sibling tools like mouse_move (which moves without pressing) and mouse_click (which clicks at a point). The mention of coordinate computation from screenshot metadata further defines the tool's scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for drag gestures but does not explicitly contrast it with alternatives or state when not to use it. However, it does provide a critical usage tip: coordinates are logical points derived from the screenshot tool's bounds_pt and px_per_pt metadata, which aids correct invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Move the mouse cursor. Coordinates are logical points, origin top-left of the main display. Compute them from the screenshot tool's bounds_pt and px_per_pt metadata.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| clamp | No | ||
| duration_ms | No | ||
| require_fresh_screenshot | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses important behavior about coordinate system and origin, but omits other behavioral traits such as clamping behavior, animation duration, and the require_fresh_screenshot parameter. These are present in the schema but not described, leaving gaps for the agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the action, and every sentence adds value. The coordinate guidance is concise and directly relevant. No fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 5 parameters, no output schema, and no annotations, the description is incomplete. It covers the coordinate system well but omits details about optional parameters and when to use alternatives. The context is sufficient for basic movement but not for advanced behaviors like clamping or fresh screenshot requirements.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the meaning of x and y as logical points and provides computation guidance. However, it does not explain clamp, duration_ms, or require_fresh_screenshot, leaving these parameters semantically incomplete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Move the mouse cursor' with a specific verb and resource. It distinguishes from sibling tools like mouse_click, mouse_drag, and mouse_scroll by focusing on pure movement. The additional coordinate computation guidance further clarifies its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on how to use coordinates (logical points, origin top-left, compute from screenshot metadata), but does not explicitly mention when to use this tool versus alternatives like mouse_click or mouse_drag. It implies usage for moving without clicking or dragging, but lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Scroll at (x, y) (moves there first) or at the current cursor. Positive dy scrolls up, negative down, in wheel ticks unless pixels=true.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| dx | No | ||
| dy | No | ||
| pixels | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses key behaviors: it moves the cursor to (x,y) before scrolling, and explains the sign convention for dy and the unit switch to pixels. However, it omits any mention of dx (horizontal scroll), which is a significant behavioral gap given no annotations are provided to cover it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, tightly-worded sentence that front-loads the action and packs in essential details without redundancy. Every phrase adds value, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description needs to cover all relevant behavioral and parameter details. It covers several aspects but misses dx entirely, and doesn't mention return values or edge cases (e.g., if both dx and dy are provided). This incompleteness leaves the tool under-specified for an agent to use correctly in all cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description carries the full burden for parameter meaning. It explains x/y (optional coordinates, moves there first), dy (vertical direction and units), and pixels (unit toggle). But dx is completely undocumented, so an agent would not know it represents horizontal scrolling. This is a notable omission.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool scrolls the mouse at specified coordinates or at the current cursor, using a specific verb ('Scroll') and resource (mouse). It distinguishes this from sibling tools like mouse_move or mouse_click by focusing on the scroll action and its coordinate behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use coordinates versus current cursor (if x/y are omitted) and explains direction semantics. However, it does not explicitly exclude alternatives or mention when not to use the tool, so it's clear guidance but not fully explicit about alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Capture the screen (or a region, in points). The response includes bounds_pt and px_per_pt: point = bounds_pt.origin + pixel / px_per_pt. Take a screenshot before any coordinate action.
| Name | Required | Description | Default |
|---|---|---|---|
| fresh | No | ||
| format | No | png | |
| region | No | ||
| max_dim | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It usefully discloses the coordinate conversion details (bounds_pt, px_per_pt and the formula), which is critical behavioral context. However, it does not explain the behavior of parameters like 'fresh' (caching), 'format', or 'max_dim', nor any side effects or permissions. The disclosure is partial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences. The first states the core action, the second provides the essential coordinate formula, and the third gives a usage guideline. Every sentence adds value and there is no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 4 parameters, no annotations, and no output schema, the description provides the most critical information (purpose and coordinate mapping) but leaves parameter semantics unexplained. It is enough for a basic screenshot use case but not fully complete for an agent to leverage advanced features like fresh or max_dim.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It only hints at the 'region' parameter via 'or a region, in points,' and the coordinate formula indirectly relates to that. It fails to explain 'fresh', 'format', and 'max_dim' parameters, which are left entirely to the schema (which offers little beyond names and default values).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Capture the screen (or a region, in points).' It uses a specific verb ('capture') and resource ('screen'), and the parenthetical 'or a region' distinguishes it from screen-wide capture tools. This is distinct from sibling tools like get_ui_tree or find_text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear usage directive: 'Take a screenshot before any coordinate action.' This tells the agent when this tool is appropriate. However, it does not explicitly mention when not to use it or name alternatives, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Check an expected outcome after acting. expect = {type: 'text_present'|'text_absent'|'region_changed'|'region_unchanged'|'cursor_at'|'window_present'|'window_gone'|'clipboard_contains'|'all_of'|'any_of', ...params, children?}. region_changed/unchanged need baseline_screenshot_id from an earlier screenshot response. text_present/text_absent require an exact (case-insensitive) substring match, not fuzzy. clipboard_contains never returns clipboard content, only matched/clipboard_len.
| Name | Required | Description | Default |
|---|---|---|---|
| expect | Yes | ||
| baseline_screenshot_id | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears the full burden and delivers key behaviors: exact substring matching, the baseline_screenshot_id requirement, and that clipboard_contains only returns matched/clipboard_len, not clipboard content. It lacks an overall return format description, but is substantially transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the purpose. The list of expect types is dense but every sentence adds value. Slight formatting improvement could help, but it is not verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
While the description covers multiple expect types and specific behaviors, it does not specify the tool's overall return value (beyond clipboard_contains) or provide an example of the expect object shape. Without an output schema, this leaves some ambiguity for a complex tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains the `expect` object structure with allowed types and mentions baseline_screenshot_id for region checks. This adds critical meaning beyond the bare schema, though a concrete example would improve clarity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states a specific purpose: 'Check an expected outcome after acting.' It also lists the supported check types, distinguishing it from sibling action/observation tools. No tautology is present.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'after acting' gives clear usage context. It also provides specific prerequisites, such as 'region_changed/unchanged need baseline_screenshot_id from an earlier screenshot response' and clarifies that text matching is exact and case-insensitive. However, it does not explicitly name alternative tools or when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Wait for a condition: {type: 'duration', ms} | {type: 'text_present', text, region?} | {type: 'screen_stable', quiet_ms}. Timeout returns met=false (an answer, not an error). text_present requires an exact (case-insensitive) substring match, not fuzzy matching.
| Name | Required | Description | Default |
|---|---|---|---|
| condition | No | ||
| timeout_ms | No | ||
| duration_ms | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description discloses key behaviors: timeout returns met=false (not an error), and text_present requires exact case-insensitive substring matching. It also specifies optional region. This goes beyond a simple statement of effects, though it doesn't cover every edge case.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences deliver the core functionality, condition formats, timeout behavior, and matching rule without wasted words. The structure uses a compact notation that is readable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description provides enough detail to use the tool correctly for its listed conditions and timeout behavior. Missing elements include a success return description and explicit usage comparisons, but the core semantics are covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, and the description compensates by explaining the condition object variants ({type: 'duration'}, {type: 'text_present'}, {type: 'screen_stable'}). However, it does not reference the top-level duration_ms parameter or explicitly define timeout_ms semantics, leaving some schema properties under-explained.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool as a wait operation with three explicit condition types, making its purpose specific. However, it doesn't explicitly contrast with sibling tools like verify or find_text, so it falls short of full differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description conveys when to use the tool (to wait for a duration, text, or stable screen) but provides no explicit guidance on choosing it over siblings like find_text or verify. Usage is implied rather than prescribed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Focus (raise) a window by window_ref, or by app and/or title_match. Stale refs are re-resolved by pid + fuzzy title.
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | ||
| window_ref | No | ||
| title_match | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the behavioral transparency burden. It discloses a meaningful trait: stale window refs are re-resolved by pid + fuzzy title. This adds value beyond the schema. Minor gaps include no mention of failure behavior or side effects, but for a focus/raise action this is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact: two sentences that front-load the action and include the key fallback behavior. No redundant words or repetition of schema fields.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's relative simplicity (no output schema, no annotations), the description covers the core behavior and adds the stale-ref handling detail. It could mention failure cases or prerequisites, but as a standalone description it is sufficiently complete for an agent to invoke correctly in most scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It partially does by explaining the roles of window_ref, app, and title_match ('by window_ref, or by app and/or title_match'). However, it lacks details on expected formats (e.g., app name vs bundle ID, title_match pattern), so it only partially compensates for the absent schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Focus (raise)') and a clear resource (a window), and specifies the two modes of selection: by window_ref or by app/title_match. This distinguishes it from sibling tools like window_list and window_manage, which have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context on when to use the tool (to focus/raise a window) and explains the two input methods (ref vs app/title_match). However, it doesn't explicitly mention when not to use it or compare it to alternatives like window_manage, so it falls short of a direct exclusion or alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
List windows (optionally filtered by app bundle id or name). Returns window_ref handles for window_focus/window_manage.
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | ||
| on_screen_only | No |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that it lists windows optionally filtered by app bundle id or name, and that it returns window_ref handles for subsequent window_focus/window_manage operations. While it doesn't mention read-only nature explicitly, 'List' implies a non-mutating behavior. Some details like on_screen_only behavior or permissions are omitted, but the core behavior is transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences long, front-loaded with the main action, and every sentence provides useful information. No redundant or vague wording.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations, output schema, and schema descriptions, the description is quite complete: it indicates the purpose, filtering capability, and return value purpose. The only minor gap is the lack of explicit mention of the on_screen_only parameter's effect, but the overall tool behavior is clear for a simple list operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the 'app' parameter by stating it can be a bundle id or name, adding value beyond the schema. However, the 'on_screen_only' parameter is not mentioned at all, and with 0% schema description coverage, the description must compensate for both parameters. It only partially does so.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('List windows') and resource, and distinguishes it from sibling tools like window_focus and window_manage by noting it returns handles for those tools. It also indicates the optional filtering by app.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool: when you need to enumerate windows and obtain window_ref handles for focusing/managing. It does not explicitly exclude alternatives, but no other sibling tool offers this functionality, making the usage context clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
move/resize/minimize/unminimize/maximize/close a window. close may trigger 'Don't Save' dialogs and needs confirmation under the default policy.
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | ||
| bounds | No | ||
| window_ref | Yes |
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description reveals a non-obvious behavior: close may trigger 'Don't Save' dialogs and requires confirmation under default policy. Given no annotations exist, this adds valuable safety information, though other actions' side effects are not detailed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the action list, with a concise caveat. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the main actions and a key caveat, but lacks parameter details (e.g., bounds needed for move/resize) and any mention of how window_ref is obtained. For a multi-action tool with no annotations or output schema, additional context would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It only repeats the action enum and mentions close, but does not describe window_ref or bounds semantics or when they are required.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly enumerates the window management actions (move, resize, minimize, etc.) and identifies the target resource (a window). It distinguishes from sibling tools like window_focus and window_list by covering a distinct set of state-changing operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance is provided on when to use this tool versus alternatives such as window_focus or mouse_drag. The description mentions caveats for close but does not state usage context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
v0.1.0app_closeapp_listapp_openclipboard_getclipboard_pasteclipboard_setexecute_sequencefind_textget_stateget_ui_treekey_presskeyboard_typemouse_clickmouse_dragmouse_movemouse_scrollscreenshotverifywaitwindow_focuswindow_listwindow_manageMost tools target distinct resource-action pairs (mouse, keyboard, window, app, clipboard). The observation/verification trio (find_text, wait, verify) has overlapping text-matching concepts, but descriptions clearly delineate fuzzy search vs exact wait vs assertion, so agents can select correctly.
The set uses a consistent underscore style but mixes noun-verb (window_list, mouse_move) with verb-noun (get_state, find_text) and bare verbs/nouns (wait, screenshot). Within each domain (mouse, clipboard, window, app) the pattern is consistent, so it's predictable overall.
22 tools is on the high end, but the server's scope is broad GUI automation (input, observation, window management, clipboard, verification), so each tool covers a distinct capability and the count is justifiable.
The tool surface covers the full interactive loop: observe (screenshot, get_ui_tree, find_text), act (mouse, keyboard, clipboard), manage windows/apps, and verify outcomes (wait, verify). No obvious dead ends; sequences allow batching.
Unclaimed servers have limited discoverability.
If you are the server author, to access and configure the admin panel.
MCP gateway with runtime security policy, tool-call-level control, and audit of agent actions.
Zero-secret MCP gateway for AI agents: risk-scored, audited calls with human-in-the-loop approval.
MCP server connecting AI agents to 100+ apps (Gmail, Slack, Notion, GitHub) via one-click OAuth.
A paid remote MCP for AI agent browser approval MCP, built to return verdicts, receipts, usage logs,
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/yuvitbatra/hands-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server
