clawtouch-mcp
The clawtouch-mcp server turns any MCP-compatible AI agent into a physical HID controller, enabling hardware-level mouse and keyboard input over a USB-connected ClawTouch device or Raspberry Pi Pico 2 running ClawTouch firmware.
Mouse Control
hid.click— Click at an absolute or relative coordinate (left/right/middle, single/double, optional glide animation)hid.move— Move the cursor to a position (absolute or relative, optional glide)hid.hover— Move to a position and idle for a duration (useful for tooltips/hover states)hid.scroll— Scroll the mouse wheel up (positive) or down (negative)hid.mouse_button_down/hid.mouse_button_up— Press/release a mouse button independently (e.g., for drag operations)hid.drag— Drag from one coordinate to another while holding a button
Keyboard Control
hid.type— Type a string as physical keyboard input (US layout, ASCII; max 4096 chars)hid.key— Press a named key or shortcut (e.g.,enter,ctrl+c,alt+tab)hid.key_press/hid.key_release— Hold or release a key/shortcut (e.g., hold Shift for multi-select)hid.hold_key— Press a key, wait a duration, then release
Safety & Batch Operations
hid.release_all— Panic stop: instantly release all held keys and mouse buttonshid.batch— Execute a pre-planned sequence of up to 10 HID actions in a single call, with per-op delays and stop-on-error control
Screen Capture (opt-in)
hid.screenshot— Capture the primary monitor as JPEG or PNG; only available when the server is started with--allow-screenshot(disabled by default for privacy)
Device Diagnostics
device.list— List candidate HID board serial ports on the hostdevice.info— Show active connection details including screen resolution, port, and serial number
The server supports both local deployment (agent and target on one PC) and cross-host deployment, integrates with MCP-compatible clients (Claude Desktop, Cursor, OpenClaw), and enforces safety limits including coordinate clamping and a 20 ops/sec rate limit.
English | 简体中文
clawtouch-mcp
Give your AI agent real hands. An MCP server that turns any MCP-compatible client — Claude Desktop, Cline, Continue, Cursor, OpenClaw, Hermes Agent and any other — into something that can move a real mouse and press real keys through a USB HID device.
What is this?
A standalone Python process that speaks Model Context Protocol (MCP) over
stdio and exposes mouse / keyboard primitives — hid.click, hid.type,
hid.scroll, key combos, hid.screenshot — to whatever AI agent you already
use. Under the hood it talks over USB serial to a ClawTouch HID device (a
Raspberry Pi Pico 2 running the open ClawTouch HID firmware, or any
turnkey ClawTouch box) and translates each tool call into a real USB HID report.
The target OS sees a genuine physical keyboard and mouse — input arrives on
the same driver path as any plugged-in peripheral, not as a software-injected
synthetic event.
📦 MIT-licensed. No ClawTouch backend, no LLM, no agent loop on top — just the raw HID plumbing so other agent stacks can talk to real hardware.
⚠️ This gives an agent real keyboard / mouse reach over a machine — the same reach as a person at the keyboard. Read Safety first.
Related MCP server: computer-use
Why hardware HID?
Software automation (PyAutoGUI, OS-level input APIs, multimodal click-the-screen models) injects synthetic input events into a session — which requires an agent process running on the target machine, in that user session, with focus. A USB HID peripheral works the other way around: it emits real HID reports that travel the standard OS HID driver stack, exactly like a plugged-in keyboard or mouse. The OS recognizes the Pico natively as a standard USB HID class device and needs no mouse / keyboard driver and no HID agent process on the input side of the target. That difference is the whole point of this project — every other section below just builds on it.
Local mode is the common case (agent + clawtouch-mcp + Pico + the screen
all on one PC; the clawtouch-mcp process lives there because it's the agent's
host, but the input side needs no driver). Cross-host control — agent on one
machine driving a target on another over USB HID — is an additional capability
the same hardware unlocks; see Deployment modes.
Good for:
Kiosks / locked-down machines — drive a machine you can't (or won't) install software on; nothing extra runs on the input side.
Accessibility — let a user drive their own computer via an agent issuing HID commands, without fighting per-app synthetic-input compatibility.
Compatibility testing — verify your software handles external HID input correctly, which can differ from injected synthetic events.
Cross-host RPA / test rigs — an agent on your dev laptop drives an industrial PC, an offline test target, or a QA-lab phone, with no agent on the target (visual feedback needs a separate path — see Deployment modes).
Not for:
Mass account creation / multi-account operations — a single-host tethered peripheral is structurally a poor fit; one device drives exactly one target, and to drive ten machines you buy ten devices.
Application-specific scripted shortcuts (selectors, fixed-flow scripts for a particular site or app). Those belong in agent / RPA frameworks built on top of this primitive layer, not in this layer itself.
For standard desktop apps (browser, IDE, office suites) the software-only path is already enough — the hardware is just an extra option there, not a requirement. Its irreplaceable value is the cases above, where the target can't host an agent, must show the OS a genuine physical HID device, or has to be driven across machines. For the compliance boundary on the "not for" cases, see Acceptable use.
Quickstart
⚠️ Before you start: read Safety — a connected agent can operate this machine like a person at the keyboard.
Install
pip install clawtouch-mcp # minimal (serial only)
pip install 'clawtouch-mcp[screenshot]' # + mss + Pillow (JPEG, Retina resize)
pip install 'clawtouch-mcp[screenshot-min]' # mss only — no native deps, works
# under hardened-runtime hosts
pip install 'clawtouch-mcp[window]' # pyobjc — macOS needs it for
# screen.windows; no-op elsewherePlatform-specific setup guides (recommended on first install):
Windows —
docs/windows-setup.md: dual COM port enumeration, VS Code Claude extension.mcp.jsonconfig, full window restart required, display-scaling notes.macOS —
docs/macos-setup.md: Keyboard Setup Assistant dialog on first plug-in, dual USB-CDC ports, Screen Recording permission, Pinyin IME punctuation gotchas.
Run
# 1. Auto-detect HID board AND auto-detect screen size (v0.2.3+)
clawtouch-mcp
# 2. Explicit port (Windows), screen still auto-detected
clawtouch-mcp --port COM7
# 3. Pin screen size manually (e.g. clamp to one monitor in a multi-monitor setup)
clawtouch-mcp --screen 1920x1080
# 4. No hardware — everything is logged, nothing moves (dev/CI mode)
clawtouch-mcp --mock --log-level INFOv0.2.3+ auto-detects the primary monitor's physical pixel size on startup so coordinates clamp to the actual screen rather than a hard-coded
1920x1080. Usedevice.infofrom your MCP client to see what was detected (screen.sourceis"detected"/"explicit"/"unset").
Use with Claude Desktop
Add to ~/Library/Application Support/Claude/claude_desktop_config.json
(macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):
{
"mcpServers": {
"clawtouch": {
"command": "clawtouch-mcp",
"args": ["--port", "COM7", "--screen", "1920x1080"]
}
}
}Restart Claude Desktop. You should see clawtouch show up in the MCP server
list with 16 tools available (14 HID + 2 device; +2 if you pass
--allow-screenshot, which enables hid.screenshot and screen.windows). Try:
Take a screenshot of my screen, find the search box, click it, and type "hello world".
(Requires --allow-screenshot to enable the hid.screenshot tool — off by
default for privacy.)
Other MCP clients
Copy-pasteable config for 7 verified clients (Claude Desktop / Code,
Cursor, OpenClaw, Hermes Agent, ChatGPT Desktop / Codex CLI,
Cherry Studio, Trae IDE) — see
examples/integrations/INTEGRATIONS.md.
PRs adding new clients welcome.
Deployment modes
Is the agent on the same machine as the screen? clawtouch-mcp covers the input side only (agent tool call → HID report → real input). The visual side (agent reads the screen to decide what to do next) is not in this repo — how you wire the two sides together depends on where the agent runs.
Local mode — the common case. agent + clawtouch-mcp + Pico + the controlled screen all on one PC. hid.screenshot captures that same screen, so the visual feedback loop closes naturally; the Pico is a standard USB HID device needing no driver. Good for accessibility, single-machine RPA, compatibility testing, in-machine kiosk self-service.
Cross-host mode — input supported, visual is your problem. agent + clawtouch-mcp on machine A; the Pico and the controlled screen on machine B. This repo fully covers the input side (A → B over USB HID), but hid.screenshot still captures A's screen, not B's — HID carries input one-way only; reverse screen capture isn't in the spec. Pick a visual path: HDMI capture card (B stays truly software-free, needs capture hardware) · VNC / RDP (open, no vendor lock-in, but B is no longer software-free) · API / log verification (check progress at checkpoints, not real-time; fixed-flow RPA only) · blind operation (pre-baked command sequence, no feedback; fully deterministic macros only). Good for industrial PCs that can't run a modern OS, strictly isolated embedded test targets, QA-lab phone farms.
Safety
Read this before connecting an autonomous agent. The runtime limits above are flood / typo guards, not a security boundary against a misbehaving agent.
Runtime safety limits
Coordinates clamped to
--screen WxHso an agent can't move the mouse to bogus pixel positions.Typed text capped at 4096 chars per call.
hid.typeis for ASCII / US-keyboard-layout text. Control characters (newline / tab / etc.) are stripped by default so an agent's multi-line draft isn't accidentally submitted — send Enter withhid.key("enter")and Tab withhid.key("tab"). Non-ASCII text (CJK, emoji) is typed through the US layout and generally will not work; drive the host IME or a clipboard path from your agent for those.All operations rate-limited to
--ops-per-sec(default 20). This counts tool calls, not individual HID reports — one call such ashid.dragor a longhid.typeemits many reports, so the effective HID-report rate is higher. It is a flood / typo guard, not a security throttle.hid.screenshotis disabled unless you pass--allow-screenshot.hid.release_allexposed for use as a panic-stop tool from the agent.
What an agent connected to this can do
clawtouch-mcp turns your agent's tool calls into real USB HID input — the
same property that makes the legitimate use cases work (kiosks, accessibility,
compatibility testing, cross-host RPA) carries a symmetric risk:
An autonomous agent connected here has, in practice, the same reach over the
host as a person sitting at the keyboard. It can open any application, run
commands in a terminal, install or remove software, and read, move, or delete
files. Because the input arrives as ordinary HID, a confirmation prompt is not
by itself a reliable barrier — treat any consent dialog as something the agent
may act on. clawtouch-mcp does not inspect intent or content; it
faithfully forwards each call to the hardware.
This can happen without you intending it, because the agent decides what to do. The usual triggers:
Prompt injection. Untrusted text the agent reads off the screen, a web page, or an image can carry instructions that override yours.
Model error. The model misunderstands the task and acts on the wrong window, file, or button.
Over-broad autonomy. The more open-ended the task and the fewer the checkpoints, the larger the blast radius.
This is an unintended failure mode, not a supported use — deliberately
using HID input to defeat a system's security controls is out of scope under
Acceptable use below. It is also distinct from software bugs in
SECURITY.md: none of those cover an agent acting against your
own intent. The MIT "AS IS / no warranty" clause is a liability disclaimer, not
an informed-risk notice; responsibility for running an autonomous agent safely
sits with you as the deployer. This notice is provided for information only —
it does not modify, narrow, or expand the MIT License, create any warranty or
duty of care, or shift liability to Tinqiao; the MIT no-warranty / no-liability
terms continue to govern in full.
Operator mitigations
Treat an agent driving HID input like giving a capable but not-fully-trusted operator real hands on the machine. Recommended:
Use a dedicated, wipeable machine (or a VM / container) — not your primary computer. Local mode puts agent and target on one PC: convenient, but the largest blast radius; prefer a separate machine where you can.
Run under a least-privilege OS account, never as administrator / root — the agent inherits whatever that account can do.
Keep secrets and logged-in accounts off the target — no saved passwords, no authenticated sessions, no credentials in the prompt.
Keep a human in the loop for consequential or irreversible actions (installing/deleting software, sending messages, financial transactions, agreeing to terms); don't leave the agent unattended on open-ended tasks.
Isolate the network (e.g. a domain allowlist) to limit exposure to malicious or injection-bearing content.
Treat everything read from screen or web as untrusted input and keep it away from sensitive data and actions.
Keep a panic stop reachable.
hid.release_allreleases every held key and button from the agent side; physically unplugging the HID device's USB cable is the most reliable stop and removes the agent's input path entirely.
If you deploy this on behalf of others (accessibility, managed RPA), inform those end users of these risks and obtain their consent.
Tools
Seventeen tools register: fourteen always-on hid.* input tools, plus
hid.screenshot (opt-in — off unless you pass --allow-screenshot), plus
two read-only device.* diagnostics. That matches the startup log line
14 HID tools + 2 device tools registered (the --allow-screenshot flag adds
hid.screenshot on top, for 17).
Tool | Since | Purpose |
| v1.0 | Click at (x, y) |
| v1.0 | Move the mouse to (x, y) |
| v1.0 | Move to (x, y), then idle |
| v1.0 | Type a UTF-8 string |
| v1.0 | Wheel scroll up / down |
| v1.0 | Press a named key or shortcut ( |
| v1.0 | Panic stop — release every held button and key |
| v1.1 | Press a mouse button without releasing (drag start) |
| v1.1 | Release a held mouse button (drag end) |
| v1.1 | Drag from one point to another while holding a button |
| v1.1 | Press a key/shortcut without releasing |
| v1.1 | Release a held key (no args = release everything) |
| v1.1 | Press, wait, then release |
| v0.4.0 | Run a pre-planned sequence of ≤10 HID actions in one call (strict order) |
| v1.0 | Screenshot a monitor or a |
| v0.5.0 | List visible top-level windows with titles and screen rectangles, so a capture can be cropped to one window. On Windows each entry also carries |
| v1.0 | List candidate HID board ports |
| v1.0 | Active connection info |
Coordinates & behavior. Click / move / hover are absolute by default:
the server queries the OS cursor position (Win32 / CoreGraphics / X11), computes
the offset to your target, and sends a relative delta to the firmware — so
{"x": 640, "y": 360} lands at that screen pixel. Pass relative=true to skip
the OS query and send a raw pixel delta instead. Where the OS cursor can't be
read (Wayland, or any OS-query failure) the call returns an explicit error —
it never silently guesses and clicks the wrong place. hid.drag composes
mouse_button_down → glided move → mouse_button_up; the v1.1 button/key
hold pair (mouse_button_*, key_press / key_release, hold_key) maps onto
the Computer-Use Anthropic (CUA) action set. hid.batch runs a short,
pre-planned list of these actions (≤10) in one call, in strict order —
a transport convenience for action lists you already know (e.g. several
fixed coordinates a solver computed), not a control-flow / "act →
observe → decide" layer; for that you still issue separate calls.
Consecutive clicks are auto-spaced by a small default gap (~50 ms) so the
OS doesn't merge or drop them; override per op with delay_ms (set 0 to
opt out).
Tool selection. The server ships built-in selection guidance so an agent
reaches for physical HID only when it's the right answer: an MCP instructions
field in the initialize response, plus a per-tool HID_PREFIX prepended to
every hid.* description (so the cue survives even if a client ignores the
server-level field). Both say the same thing — prefer hid.* only as a
fallback, when no file / browser / OS API can do the job, or when the user
explicitly asks for physical keyboard / mouse input. The read-only device.*
tools carry no prefix.
Examples
Most agents reach clawtouch-mcp through an MCP client (Claude Desktop / Code, Cursor, and others) — copy-pasteable configs for the verified clients are in examples/integrations/INTEGRATIONS.md.
If you're building your own Computer Use loop instead, examples/computer_use/ has two reference implementations that route agent actions through ClawTouch HID:
Claude Computer Use → HID —
client.beta.messages.streamwith thecomputer_20251124toolOpenAI CUA → HID — Responses API with
computer-use-preview
For per-application LLM guidance, clawtouch-skills is a companion repo of markdown operator manuals an LLM can load before driving a specific app. Skills are soft guidance — the LLM still decides what to do.
See it in action
Start the server against your bridge, then any MCP client (Claude
Desktop, Cline, or your own loop) speaks plain MCP tools/call over
stdio. Each call becomes a real USB-CDC frame to real hardware —
nothing synthetic.
$ clawtouch-mcp --port COM7
[INFO] connected to Pico 2 on COM7 (serial: E660ABCD12345678)
[INFO] screen auto-detected: 2560x1440 (Windows SM_CXSCREEN/SM_CYSCREEN)
[INFO] 14 HID tools + 2 device tools registered; listening on stdio
# client → server : one click, then one typed string
# (the cursor and keys actually move)
→ tools/call hid.click {"x": 640, "y": 360}
← result "clicked at (640, 360)"
→ tools/call hid.type {"text": "Hello from MCP"}
← result "typed 14 chars in 0.42s"Acceptable use
This server is built for legitimate uses — accessibility, RPA, test automation, cross-machine workflows where the target machine must stay clean. This project does not support, document, or assist with use cases that:
Bypass, evade, or interfere with any target platform's anti-fraud, anti-abuse, rate-limiting, or risk-control measures.
Operate accounts the user does not lawfully own or have explicit authorization to operate.
Are prohibited by the target application's Terms of Service in the user's jurisdiction.
Violate applicable law — including, but not limited to, PRC Anti-Unfair Competition Law Art. 13 (the Internet sector specific provision; promulgated 2025-06-27, effective 2025-10-15) covering improper means — including circumventing technical management measures — to acquire or use another operator's data; Personal Information Protection Law; Cybersecurity Law; and equivalent laws in other jurisdictions.
These statements describe the scope of our maintainer support and documentation — they are not additional restrictions on the MIT License, which continues to govern all use, modification, and redistribution of the source code. Users are independently responsible for evaluating their specific use case against applicable laws and the target platform's ToS.
Content generation — out of scope
clawtouch-mcp exposes hardware HID actions (mouse / keyboard / scroll
/ key combos / screenshot) as MCP tools. It does not generate,
synthesize, recommend, or otherwise produce text, image, audio, or
video content. The calling LLM agent is the content-generating party
and is solely responsible for any generated content and for
compliance with any content-labeling or content-moderation
obligations applicable in its jurisdiction (e.g. PRC AI Generated
Content Labeling Measure effective 2025-09-01).
Hardware
This server can talk to:
ClawTouch HID device — turnkey hardware, drop-shipped, plug-and-play. Order or get a sample at clawtouch.cn.
Any RP2350 board running clawtouch-hid — the OSS firmware + protocol module (wire epoch 1, frozen envelope) live in their own public repo. Buy a Pico 2 (~$8), flash the firmware, you're done.
The wire protocol is the same for both — the server doesn't care which one it talks to.
FAQ
Does this need a ClawTouch account / API key / cloud service? No. This server only speaks USB serial to the HID board. There's no network call. No data leaves your machine.
Can I use this without buying ClawTouch hardware? Yes — buy an $8 Raspberry Pi Pico 2, flash the open-source clawtouch-hid firmware, and the server will talk to it the same way as the turnkey device.
How is this different from the closed-source ClawTouch desktop app?
This MCP server is the bottom HID primitive layer. The desktop product
is a separate closed-source agent on top of the same hardware; contact
support@tinqiao.com for details.
Is there a JavaScript / TypeScript version?
Not yet. clawtouch-bridge-sdk (Python + Node) is planned — see the
Open source roadmap.
hid.screenshot fails with ImportError: dlopen(... _imaging ...) code signature ... different Team IDs?
Your host Python has a hardened runtime with library validation, which only
allows native extensions signed with the host's own Team ID — so it blocks
Pillow's _imaging (some bundled/launcher Pythons do this). The server
auto-detects this and falls back to the no-native mss-png backend,
returning a PNG with an explanatory note in the metadata — you don't have to
do anything. mss itself is pure-Python (ctypes → CoreGraphics, a platform
framework that's exempt), so it loads fine; only Pillow's compiled extension
is blocked. To force it, pass --screenshot-backend mss-png or install the
lean pip install 'clawtouch-mcp[screenshot-min]' (mss only). To keep Pillow
(JPEG + higher-quality resize), run clawtouch-mcp from a Python without
library validation, or grant the host Python the
com.apple.security.cs.disable-library-validation entitlement. Either way the
screenshot's backend field and scale_x/scale_y tell the agent what it
got — always divide click coordinates by scale_x/scale_y.
Related work
The MCP / Computer-Use ecosystem already has projects that hand an LLM agent control of a desktop, in two camps. Software-only MCP servers on the target PC — domdomegg/computer-use-mcp, AB498/computer-control-mcp, the various mcp-pyautogui implementations, and ByteDance's UI-TARS — call PyAutoGUI / OS input APIs in-process: lowest friction, but the agent shares the target's OS / session / focus, and a crash disrupts the user's actual desktop. Hardware-bridge servers decouple the two: sunasaji/mcp-serial-hid-kvm (a CH9329 / CH9350L USB-HID ASIC plus capture card) is the closest direct peer in architecture, and CMU's HIDAgent (Bigham et al., 2026-01; < $30 RP2040 + HDMI-to-USB + CH340 serial bridge, shipped as a Python library) the closest academic peer. clawtouch-mcp follows the same decoupling but pairs with the open-firmware clawtouch-hid stack, so the wire protocol is user-extensible and the firmware is auditable — not a fixed-function ASIC.
ClawTouch's irreplaceable edge is the genuine hardware HID path: the OS sees a real physical keyboard / mouse. In local mode — the common case — that real HID plus zero driver on the input side is exactly what makes it work for accessibility, compatibility testing, and apps that reject synthetic input; if you only need synthetic input on one machine and the app doesn't care where input comes from, the software-only servers above are simpler. Cross-host mode is an additional capability on top: it can drive a target that can't host an agent or must stay physically isolated — something a software-only server can't do at all.
Open source roadmap, contributing & license
Open-core model. Hardware and protocol primitives are open; the integrated commercial product stays closed.
Component | Status |
clawtouch-mcp (this repo) | ✅ Released |
clawtouch-hid — firmware + protocol module, wire epoch 1 | ✅ Released |
clawtouch-skills — markdown skill files for LLM agents | ✅ Released |
clawtouch-bridge-sdk — Python + Node HID SDK | 🔵 Future |
Backend / desktop app / adapters / vision models | 🔒 Closed — |
Contributing
PRs welcome for: new MCP tools mapping to existing HID primitives, bug fixes, client-integration examples, doc improvements, non-English README translations.
Not taking PRs for: agent-loop logic or application-level features (intentionally out of scope — see Acceptable use) or adapters for specific applications (those live in the closed-source desktop app).
clawtouch-mcp is maintained by Tinqiao Technology — the team behind ClawTouch (clawtouch.cn).
License
MIT © Tinqiao Technology (Beijing) Co., Ltd. — see LICENSE (English, authoritative) and LICENSE.zh-CN.md (non-official Chinese translation, for reference).
Third-party dependencies and their licenses are listed in NOTICE. Trademarks (ClawTouch, Tinqiao, and third-party marks referenced in this repository) are covered separately in TRADEMARKS.md — the MIT License does not grant any trademark rights.
For commercial deployments at scale, enterprise support, or OEM hardware
discussion: support@tinqiao.com.
Architecture overview
flowchart LR
A["<b>LLM Agent</b><br/><sub>Claude Desktop / Cline /<br/>Cursor / OpenClaw / Hermes / ...</sub>"]
-->|"stdio<br/>JSON-RPC<br/>MCP 2024-11-05"| B["<b>clawtouch-mcp</b><br/><sub><i>this repo</i><br/>MCP server + 14 HID + 2 device tools</sub>"]
B -->|"USB-CDC<br/>v1.0 framed bytes"| C["<b>Pico 2</b><br/><sub>+ ClawTouch HID firmware<br/>(RP2350 / CircuitPython)</sub>"]
C -->|"USB HID<br/>reports"| D["<b>Target OS</b><br/><sub>Windows / macOS / Linux<br/>standard HID driver stack</sub>"]
classDef this fill:#fef3c7,stroke:#d97706,stroke-width:3px,color:#78350f;
class B this;This repo is the MCP-server hop: it translates MCP tool calls into framed bytes over USB-CDC; the firmware turns them into standard USB HID reports. For how it fits the larger Perception → Decision → Action loop and how the closed-source desktop app layers on these open HID primitives, see the official technical documentation: system architecture & data flow and data security & compliance.
Available Tools
16 toolsdevice.infoB
Active bridge's connection + sequence info.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It notes the output nature (connection and sequence info) but fails to mention read-only nature, side effects, or permissions. This is insufficient for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, zero waste, front-loads the purpose. Perfectly concise for a trivial tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema or annotations, the description should provide more context about the return value beyond 'connection + sequence info'. It is minimally complete but leaves ambiguity about what exactly is returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters are defined, and the description adds no additional semantic information. Per guidelines, with 0 parameters and full schema coverage, baseline is 4. The description is adequate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states it returns 'connection + sequence info' for an 'active bridge', implying a retrieval function, but lacks a clear verb like 'Get' or 'Retrieve'. It distinguishes from device.list (list devices) and HID tools implicitly, but could be more explicit about the resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool versus siblings. The description does not mention prerequisites or situations where it's appropriate, leaving the agent to infer from context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
device.listC
List candidate Pico serial ports.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It only states the basic purpose without disclosing behavioral traits such as what 'candidate' means, permissions needed, side effects, or return format. This is insufficient for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (4 words) and front-loaded. No wasted words. However, it is slightly too terse for the lack of other context, but still efficient. Scores 4.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema, no annotations, and minimal description, important context is missing (e.g., what 'candidate' means, whether the list is dynamic or static, blocking behavior). The description is incomplete for a tool that could benefit from more detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and the schema coverage is effectively 100%. The description adds no parameter information beyond what the schema provides, which is nothing. Baseline is 3 per guidelines.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb 'List' and a clear resource 'candidate Pico serial ports', which distinguishes it from sibling tools like 'device.info' and various HID tools. It clearly communicates the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use vs. alternatives, no prerequisites, or exclusions. The context signals indicate siblings, but no comparison is made, leaving the agent to infer usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.batchA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Run a SHORT, PRE-PLANNED sequence of HID actions (max 10) in ONE call, in strict order. This is a transport convenience for an action list you ALREADY know — e.g. clicking several fixed coordinates a solver has computed — collapsing N tool round-trips into one. It is NOT an orchestration / control-flow layer: no branching, no reading a result mid-sequence, no looping. For 'act → observe → decide → act' you still issue separate calls (an action that depends on an earlier action's outcome cannot be pre-planned into a batch).
Each op is {type, ...params, delay_ms?}. Types:
• click / move — (x, y, relative, button, double, move_ms); identical absolute closed-loop converge and ACK semantics to hid.click / hid.move.
• button_down / button_up — (button).
• key — (key, modifiers); same 'ctrl+c' shorthand as hid.key.
• type — (text).
• scroll — (delta).
delay_ms pauses AFTER that op (0–2000 ms). Omit it and click/button ops get a small default gap (~50 ms) so the OS doesn't merge or drop back-to-back clicks; non-click ops default to 0. Set delay_ms explicitly (including 0) to override.
Execution: ops run strictly sequentially. With stop_on_error=true (default) the run halts at the first op that fails; if any button/key was pressed before the stop, release_all fires so nothing stays held. Returns {ok (= every op ok), count, failed_index, stopped_early, released_all, results:[per-op dicts carrying the same fields the standalone tool returns — e.g. converged / clicked / chars]}. Held state is NOT auto-released on clean completion, so a batch may intentionally leave a button/key down for a follow-up call.
Capped at 10 ops: this drives real input and a batch cannot be interrupted mid-run (stdio is serial), so a large blind burst is refused at the boundary.
| Name | Required | Description | Default |
|---|---|---|---|
| ops | Yes | Up to 10 HID actions, executed in array order. | |
| stop_on_error | No | Halt at the first failing op (default). false = run every op, recording failures. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description fully discloses execution behavior: sequential ops, stop_on_error default with release_all on failure, delay defaults, cap at 10 ops, and no auto-release on clean completion. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with sections for usage, ops, delay, execution, return. Every sentence adds value, but could be slightly more concise. No fluff, but length might overwhelm some agents.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Comprehensive: covers all op types, delays, error handling, return structure, and constraints. No output schema needed as return format is clearly described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds meaningful context for each op type (e.g., click/move semantics identical to standalone tools, delay_ms defaults). Provides rationale for defaults, enhancing understanding beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it runs a short, pre-planned HID sequence, distinguishing from individual HID tools by collapsing multiple calls into one. It specifies the scope (max 10 ops, strict order) and contrasts with orchestration workflows.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises when to use: when other automation paths fail or user requests physical input. Also clarifies what it's NOT for (branching, reading results mid-sequence, looping), guiding the agent away from misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.clickA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Click mouse. Default semantics: (x, y) is an ABSOLUTE screen coordinate — the server queries the OS for the current cursor position (Win32 GetCursorPos / macOS CGEventGetLocation / Linux/X11 XQueryPointer via ctypes) and emits a relative move so the firmware (which is a USB Boot Mouse and only supports relative deltas) lands at the target. Pass relative=true to skip the OS query and send (x, y) directly as a pixel delta. On Wayland and on hosts where the OS cursor query fails, absolute mode returns an error and the caller must use relative=true.
Absolute mode runs a closed-loop converge (query → delta → settle, up to 10 iterations, ≤5 px tolerance) to absorb OS pointer-ballistics non-linearity (macOS scales single HID deltas ~110% in the low-speed segment, so a fire-and-forget move overshoots by 10-90 px). The returned x/y are the actual landing coordinates; target_x/target_y echo the request; converged: true means residual ≤5 px. The click only fires after the move succeeds — i.e. the cursor is confirmed within ≤5 px of target. If convergence fails (or the OS cursor query is unavailable), NO click is sent and the move result is returned unchanged (ok: false plus converged/residual_x/residual_y/hint); inspect those and retry.
Optional move_ms switches to glide mode: the move is broken into ~10 ms HID reports over N ms (linear interpolation, then a closed-loop converge pass to clean up the final landing). Default 0 = snap mode.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| button | No | left | |
| double | No | ||
| move_ms | No | Glide mode: break the move into ~10 ms HID reports over N ms (linear interp + post-slide converge). 0 = snap mode (default, instant move). | |
| relative | No | If true, x/y are pixel deltas; absolute mode is skipped. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully bears the burden of behavioral disclosure. It exhaustively covers the coordinate system (absolute/relative), OS cursor query, convergence logic with iteration count and tolerance, glide mode, click-only-after-move guarantee, error handling on convergence failure, and platform-specific behaviors (Wayland, macOS).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with front-loaded usage guidance. Every sentence provides valuable detail, though some complexity could be streamlined. Given the intricate behavior (convergence, OS queries), the length is justified. It earns a 4 for efficiently packing essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description thoroughly explains return values (x, y, target_x, target_y, converged, ok, residual_x, residual_y, hint) and error scenarios (convergence failure, absolute mode unavailability). It covers all critical aspects needed for correct invocation and handling, making it complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 33% (only relative and move_ms have descriptions). The tool's description adds significant meaning to x/y (absolute vs. relative semantics, OS queries), relative (skip absolute), and move_ms (glide mode detail). However, button and double parameters are not elaborated beyond the schema, which already provides enum and defaults. Overall, the description compensates well for the low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs a mouse click with physical HID input. It explicitly distinguishes this tool as the choice when other automation paths fail, and it details the coordinate system and modes. The verb 'click' and resource 'mouse' are unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description begins with explicit guidance on when to use this tool (when other automation paths cannot accomplish the task or user requests physical input). It explains absolute vs. relative modes and when to use relative (when absolute fails). It also advises on retrying after convergence failures, providing clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.dragA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Drag from (from_x, from_y) to (to_x, to_y) while holding the named button. Internally: absolute move to source → mouse_button_down → glided absolute move to destination → mouse_button_up. Matches Anthropic Computer Use's left_click_drag action. Useful for design / spreadsheet / file-manager workflows where 'press → drag → release' is the atomic UI gesture.
| Name | Required | Description | Default |
|---|---|---|---|
| to_x | Yes | ||
| to_y | Yes | ||
| button | No | left | |
| from_x | Yes | ||
| from_y | Yes | ||
| move_ms | No | Duration of the held-button move from source to destination. | |
| relative | No | If true, from_x/y and to_x/y are pixel deltas, not absolute coords. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the internal steps: absolute move to source, mouse_button_down, glided absolute move to destination, mouse_button_up. This gives insight into the sequence of events. However, it does not disclose potential side effects (e.g., cursor visibility, focus changes), safety considerations, or prerequisites (e.g., screen resolution). The disclosure is partial but sufficient for basic understanding.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a purpose statement and structured with internal step details. It is somewhat lengthy but every sentence adds value. It could be condensed slightly, but overall it is appropriately sized and well-organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 7 parameters, no output schema, and no annotations. The description explains the drag gesture and internal steps but does not cover return values (presumably void), error conditions (e.g., out-of-bounds coordinates), or integration with other tools. It mentions relative coordinates and duration, which is helpful. Given the complexity, it is somewhat complete but lacks elaboration on edge cases and behavior under various conditions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 29% (only move_ms and relative have descriptions). The tool description does not add meaning to parameters beyond the overall drag process. For example, from_x, from_y, to_x, to_y are not individually explained, and button only has an enum but no description of its effect. The description mentions 'named button' but does not clarify the impact of choosing left, right, or middle. It does not compensate for the low schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: drag from (from_x, from_y) to (to_x, to_y) while holding a button. It distinguishes from siblings by specifying when to use this tool over other automation paths and mentions matching Anthropic Computer Use's left_click_drag action. The verb 'drag' and resource 'HID input' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to pick this tool: 'when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.' It also lists example workflows (design, spreadsheet, file-manager). However, it does not explicitly state when not to use it or mention alternatives among the sibling HID tools, though the context implies HID simulation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.hold_keyA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Press a key, wait duration_ms, then release. Matches Anthropic Computer Use's hold_key action. Useful for scenarios where a single tap is too short — e.g. holding an arrow key to scroll a long list, or holding Space to pan in a design app.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | ||
| modifiers | No | ||
| duration_ms | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Describes the action sequence (press, wait, release) and duration range, but does not discuss potential side effects, permission requirements, or simulation fidelity beyond matching Anthropic's action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a purpose: usage context, action description, and examples. No wasted words, though slightly longer than minimal. Front-loaded with critical usage instruction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers action and usage well, but lacks output specification and error handling. With no output schema, description should hint at return behavior. Parameter explanation is partial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description adds meaning beyond schema: explains duration_ms as wait-then-release, mentions modifiers implicitly. Does not explain key values or modifiers enum. Examples help but not exhaustive.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it holds a key for a duration, specifically for physical HID input when other automation fails. Explicitly distinguishes from siblings by referencing alternative automation paths and matching Anthropic Computer Use's hold_key.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance (other automation fails, user requests physical input) and concrete examples (scrolling, panning). However, does not explicitly exclude sibling HID tools like hid.click or hid.key.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.hoverA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Move mouse to (x,y) then idle for duration_ms (no click). duration_ms is the IDLE time AFTER reaching the target; move_ms (optional) is the time spent on the move ITSELF (glide mode). Default move_ms=0 = snap mode (instant move), then idles. Absolute mode runs the same closed-loop converge as hid.click — see that tool's description for landing / convergence semantics.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| move_ms | No | Glide mode for the move itself: break into ~10 ms HID reports over N ms (linear interp + post-slide converge). 0 = snap mode (default, instant move). | |
| duration_ms | No | Idle time AFTER reaching target. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description explains idle timing, glide vs snap modes, references convergence semantics from hid.click, and discloses default behaviors. It is transparent about movement mechanics and closed-loop control, though relies on linking to another tool for full convergence detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with purpose and usage context, then concisely explains parameters. It is slightly longer due to the preamble, but every sentence contributes value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 4 parameters and no output schema, the description covers all necessary aspects: purpose, parameter behaviors, convergence reference, and usage context. It is sufficient for an agent to select and invoke correctly, though some missing details (e.g., return value) are minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 50% (x and y lack descriptions), but the description compensates by explaining them contextually. It adds significant detail for duration_ms and move_ms beyond schema descriptions, such as the idle timing and glide mode mechanics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Move mouse to (x,y) then idle for duration_ms (no click)', distinguishing it from siblings like hid.click (which clicks) and hid.move (likely without idle). The preamble also clarifies when to use physical HID input, making purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends this tool when other automation paths fail or when physical input is requested, providing good context. It does not explicitly list exclusions, but the sibling set and parameter details imply appropriate use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.keyA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Press a key or keyboard shortcut.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | Named key (enter/tab/f1…), a single character, or shortcut shorthand like 'ctrl+c' or 'ctrl+alt+l' — modifiers in the prefix are split out and combined with the modifiers array. | |
| modifiers | No | Explicit modifier list. Combined with any modifiers parsed from the key shorthand. Optional when the key field already encodes the modifiers (e.g. 'ctrl+c'). |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It only mentions 'Press a key or keyboard shortcut' but does not state whether it is a press-and-release or a press-hold, whether it blocks, or any side effects. This is insufficient for an agent to understand the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with a single sentence plus a usage guideline in brackets. It is front-loaded with the core action. However, it could be slightly more structured to separate the guideline from the action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (2 params, no output schema), the description provides a usage guideline but does not explain the life cycle of the key press (e.g., auto-release vs. hold) or the return value. This leaves gaps for an agent to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema documentation coverage is 100%, so the description adds no extra meaning to parameters beyond what is in the schema. The description merely repeats the overall action, meeting the baseline for high coverage but not adding value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The tool name 'hid.key' and description 'Press a key or keyboard shortcut' clearly state the action and resource. The description also distinguishes from siblings by specifying it's for physical HID input when other automation paths fail, and references siblings like hid.type and hid.key_press.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: 'pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.' This provides clear usage context and implicitly suggests alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.key_pressA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Press a key (or shortcut) WITHOUT releasing. Pair with hid.key_release. Useful for 'hold shift while clicking N times' multi-select patterns: hid.key_press('shift') → several hid.click → hid.key_release('shift'). For a fixed-duration hold, prefer hid.hold_key.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | ||
| modifiers | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It discloses that the key is pressed without releasing, pairs with hid.key_release, and gives a use case. Could add more about side effects or state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with purpose and usage guidance, then example. Slightly long but well-structured; every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers main use cases, distinguishes from siblings, provides example. Lacks error conditions or key format details, but overall complete for the complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description should explain parameters. It mentions modifiers list implicitly via example but doesn't detail 'key' values (e.g., special keys) or modifiers purpose beyond the enum.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it presses a key without releasing, and distinguishes from siblings like hid.hold_key and hid.key_release. It gives specific usage for multi-select patterns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly tells when to pick this tool ('when other automation paths cannot accomplish the task or user requests physical input') and contrasts with hid.hold_key for fixed-duration holds.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.key_releaseA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Release a previously-pressed key (or shortcut). Idempotent. Pass no arguments to release ALL held keys and mouse buttons (panic stop, same as hid.release_all).
| Name | Required | Description | Default |
|---|---|---|---|
| key | No | ||
| modifiers | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It states idempotency and that passing no arguments releases all held keys/mouse buttons (panic stop). However, it does not specify what happens if the key is not currently held or if only some modifiers are released.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is mostly concise but includes a bracketed usage note upfront, which adds context. The main functional description is short, but the bracketed part could be considered extraneous. Overall, it is well-structured and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple parameter set and no output schema, the description covers when to use, idempotency, and panic stop. It lacks mentioning the return value, but that is minor. For a HID release action, it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description should explain parameters. It only implies that both key and modifiers are optional to release all, but does not describe what 'key' or 'modifiers' represent (e.g., which key to release, which modifiers to release). The schema has defaults but no description of their semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool releases a previously-pressed key or shortcut. It distinguishes from the sibling tool hid.release_all by noting that passing no arguments does the same thing, providing specific verb+resource with clear scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use this tool: when other automation paths fail or when the user requests physical input. It also mentions idempotency and the panic stop behavior with no arguments, guiding appropriate usage versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.mouse_button_downA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Press a mouse button WITHOUT releasing it. Pair with hid.mouse_button_up (and hid.move in between) to compose a drag, or use hid.drag for a one-call wrapper. Matches Anthropic Computer Use's left_mouse_down action.
| Name | Required | Description | Default |
|---|---|---|---|
| button | No | left |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Clearly states the tool presses without releasing. No annotations provided, so description carries the burden. Could mention behavior if button already pressed, but sufficient for a simple HID action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise, front-loaded with purpose, then usage context and sibling references. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Complete for a simple tool with one parameter and no output schema. Covers purpose, usage, behavior, and relationships to sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and description does not elaborate on the 'button' parameter beyond the schema's enum. The description adds no additional meaning for parameter selection.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool presses a mouse button without releasing it, specifying verb and resource. It distinguishes from siblings like hid.mouse_button_up and hid.drag.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance on when to use (physical HID input as fallback or user request) and how to compose with siblings (hid.mouse_button_up, hid.move, hid.drag) for dragging.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.mouse_button_upA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Release a previously-pressed mouse button. Idempotent — releasing a non-held button is a no-op (no error). Matches Anthropic Computer Use's left_mouse_up action.
| Name | Required | Description | Default |
|---|---|---|---|
| button | No | left |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description shoulders the transparency burden. It states idempotency (no-op for non-held button) and references Anthropic's standard, providing key behavioral traits. No mention of return or side effects, but sufficient for a simple release.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured with a usage preamble and main action statement. Each sentence adds value, though the preamble could be slightly tighter. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool with no output schema, the description covers purpose, idempotency, and usage context. It mentions matching an industry standard. Could note pairing with hid.mouse_button_down, but siblings cover that.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description should compensate by elaborating on the button parameter. It only says 'mouse button' generically, adding no value beyond the schema's enum. The parameter is simple, but the description fails to explicitly list options.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool releases a previously-pressed mouse button, using specific verb and resource. It distinguishes from siblings by explicitly matching Anthropic's left_mouse_up and being the release counterpart to hid.mouse_button_down and hid.click.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The preamble provides context on when to use HID tools (when other automation paths fail or explicit physical input is requested). The idempotency note guides safe usage, but does not contrast with alternatives like hid.click for combined press-release.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.moveA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Move mouse. Default semantics: (x, y) is an ABSOLUTE screen coordinate (see hid.click for how absolute mode works under the hood, including the closed-loop convergence that absorbs OS pointer-ballistics). Pass relative=true to send (x, y) as a pixel delta directly. On hosts where the OS cursor query is unavailable, absolute mode returns an error.
Returns x/y = actual landing coordinates, target_x/target_y = original request, converged / iters for the absolute path. Optional move_ms switches snap mode (default) → glide mode; see hid.click for the trade-off.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| move_ms | No | Glide mode: break the move into ~10 ms HID reports over N ms (linear interp + post-slide converge). 0 = snap mode (default, instant move). | |
| relative | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description discloses key behavioral traits: closed-loop convergence, snap vs glide mode, error condition when OS cursor query unavailable, and return fields. It references hid.click for more details on absolute mode behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with usage context, then provides specifics on coordinates, relative mode, return values, and glide mode. Each sentence adds value without redundancy, though cross-referencing hid.click could be considered a minor duplication.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of 4 parameters, no output schema, and no annotations, the description covers key aspects: coordinate semantics, relative mode, move modes, error conditions, and return fields. It references hid.click for trade-offs, making it reasonably complete for a tool in a suite.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With only 25% schema coverage, the description adds significant meaning: explains x,y as absolute screen coordinates under default semantics, relative flag for pixel deltas, and move_ms for glide mode. It compensates for the lack of schema descriptions on other parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Move mouse' and specifies absolute vs relative coordinates, distinguishing it from other HID tools. It explicitly indicates this is for physical HID input when other automation paths cannot be used.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit guidance on when to use the tool ('pick this when other automation paths... cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input'). It also references hid.click for more details, which helps in understanding context, though it doesn't explicitly say when not to use.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.release_allA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Release every held key / mouse button (panic stop).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. States it releases all held keys/buttons, implying immediate effect. Does not mention behavior when nothing is held or any side effects. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences – a general usage prefix and the specific action. The prefix is somewhat boilerplate but concise overall. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters or output schema, the description is sufficient to understand what the tool does. Sibling tools provide context. Could mention if it affects all HID devices or just current device, but not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters in schema. Description adds no param info, but none is needed. Baseline for 0-param tool is 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states 'Release every held key / mouse button (panic stop)' – a specific verb+resource. Distinguishes from siblings like hid.key_release (specific key) and hid.hold_key (hold action).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Includes a general prefix recommending HID input when other automation paths fail or user requests physical input. For this tool, the name and description imply use as a panic stop, but no explicit when-to-use vs alternatives or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.scrollA
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Scroll the mouse wheel. Positive=up, negative=down.
| Name | Required | Description | Default |
|---|---|---|---|
| delta | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits. It adds the direction mapping for the delta parameter but omits details like scroll magnitude effects, speed, or units. The tool is nondestructive, but additional context would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively concise with two sentences. The first sentence is a longer preamble about when to use the tool, which could be separated but does not detract significantly. Overall, it is front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool with one parameter and no output schema, the description is fairly complete. It covers purpose, usage context, and parameter semantics. However, missing information about scroll behavior (e.g., lines vs pixels, limits) leaves gaps for an agent to fully anticipate the tool's effect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides no description for the delta parameter (0% coverage). The description adds meaning by explaining that positive values scroll up and negative values scroll down, which is essential. However, it does not specify units or magnitude effects, leaving some ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool scrolls the mouse wheel and defines direction with 'Positive=up, negative=down'. The preamble about selecting HID input adds context but does not directly define the tool's purpose, which is specific enough to distinguish from siblings like hid.click or hid.move.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using this tool when other automation paths fail or when physical input is requested, providing clear usage guidance. However, it does not explicitly state when not to use it, though the positive indication is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hid.typeB
[Physical HID input — pick this when other automation paths (file APIs, browser automation, OS APIs) cannot accomplish the task, or when the user explicitly requests physical keyboard or mouse input.] Type a string as if on a physical keyboard (US layout).
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must shoulder the full burden. It states basic behavior (type a string as if on a US layout keyboard) but omits details about speed, special characters, key release handling, or layout limitations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first provides context for when to use, the second states the action. It is front-loaded and contains no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and one parameter, the description is adequate for basic use but lacks details on input limitations, integration with other HID tools, and error scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It mentions 'type a string' but adds no detail beyond what the parameter name implies, such as accepted formats, length restrictions, or encoding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it types a string as physical keyboard input. It distinguishes from other automation paths but does not explicitly differentiate among HID sibling tools like hid.key for single keystrokes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises using this tool when other automation paths fail or when physical input is requested, providing clear context. However, it does not specify when to prefer other HID tools like hid.key for single keys.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
1 tool update
v0.4.6- Added
hid.batch
15 tool updates
v0.3.2- First observed
device.info - First observed
device.list - First observed
hid.click - First observed
hid.drag - First observed
hid.hold_key - First observed
hid.hover - First observed
hid.key - First observed
hid.key_press - First observed
hid.key_release - First observed
hid.mouse_button_down - First observed
hid.mouse_button_up - First observed
hid.move - First observed
hid.release_all - First observed
hid.scroll - First observed
hid.type
TDQS
Scored across 16 tools
Every tool has a clearly distinct purpose. Device tools are separate from HID tools, and within HID, each action (click, drag, hover, key press/release, etc.) has a unique tool with precise semantics, avoiding ambiguity.
All tools follow a consistent 'domain.action' pattern with snake_case for compound action names (e.g., hid.key_press, hid.release_all). The naming is uniform and predictable.
16 tools is well-scoped for a server focused on HID input. It covers a comprehensive set of operations without being excessive, and each tool serves a necessary role.
The HID toolset is thorough, covering click, drag, hover, keyboard, scroll, batch, and panic stop. Missing is a tool to explicitly connect/disconnect from the bridge, but the device.list and device.info tools imply an active connection is managed elsewhere.
Maintenance
Related MCP Connectors
MCP server for progressive tool usage at any scale (see https://klavis.ai)
One AI endpoint to search and call 22k+ MCP servers; 50+ hosted tools work instantly, no key.
- QuallaaOAuthcom.quallaa
Talk to your public-facing AI from any MCP client — Claude, ChatGPT, Cursor, Cline, Windsurf.
Host your MCP tool over streamable HTTP in one command.
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceAn MCP server that gives any AI assistant eyes and hands on your desktop — screenshots, clicking, typing, OCR, window management, accessibility-tree queries, workflow recording.5Apache 2.0
- AlicenseNot gradedqualityCmaintenanceStandalone MCP server that gives AI agents full GUI control over macOS — screenshots, mouse, keyboard, apps, clipboard, and multi-display — with zero private dependencies.18MIT
- AlicenseNot gradedqualityBmaintenanceMCP server providing AI-friendly computer-use primitives (capture, detect, click) to let LLM agents drive desktop GUI applications on Windows, macOS, and Linux.MIT
- FlicenseNot gradedqualityBmaintenanceProvides MCP tools for screen capture, HID mouse/keyboard control, system and ESP status, and local Ollama chat/vision through a local MCP endpoint.-