Computer Use
Summary: An MCP server that gives any coding agent eyes and hands on your real macOS, Windows or Linux desktop and your signed-in Chrome — reading the UI as an accessibility tree with element ids and acting in the background so your mouse stays yours.
See the screen:
list_apps,list_displays,get_app_state(accessibility tree with clickable ids like[e12],query/max_depthfilters),screenshot(with screen-origin and pixels-per-point mapping), andzoomfor full-resolution regions of small text.Act on native apps:
click,right_click,hover,drag,scroll,type_text,set_value,select_text,press_key(named keys + modifiers like cmd/ctrl),activate_app, andwait— by element id or screen coordinates.Verify in the same call: every action takes
return_state(plusstate_query) to append a fresh accessibility tree after the action, so the agent can look → act → confirm without a second round trip.Drive Chrome:
browser_open_tab,browser_list_tabs,browser_use_tab/browser_release_tab(adopt and hand back a tab the user already has open),browser_select_tab,browser_snapshot,browser_read(page text with headings,query,offset,include_links),browser_click,browser_type,browser_press_key,browser_navigate, andbrowser_close_tab/browser_close_all_tabs— all working on background tabs in the agent's own labelled tab group.Sign in safely:
browser_request_credentialsshows the user a Chrome window naming the site's real origin; typed secrets go straight into the page and never reach the model.Clipboard:
clipboard_readandclipboard_writefor plain text.Stay out of the way: on macOS the user's pointer is never moved (events are addressed to the target window); on Windows and Linux some inputs fall back to real mouse input and report
via cursor. Minimized/hidden windows can be parked on an invisible display, and results are honest about what happened.Respect user rules: per-app and per-site
allow/ask/blockpolicy, password-field typing refused by default, andCOMPUTER_USE_*flags (hide browser tools, disable the agent pointer, remote-control mode, and more).Task isolation: a stable
session_idgives each task its own tabs and cleanup, so parallel tasks and multiple MCP clients share one Chrome extension without colliding.Work with anything: standard MCP stdio server with identical tool schemas on every platform, usable from Claude Code, Codex, Cursor, MT Code or any MCP client, with no vision model required.
Allows agents to control a Linux desktop through AT-SPI and X11 accessibility trees, with element-level actions and background input; native Wayland apps support element actions but not coordinate clicks.
Allows agents to control a macOS computer through accessibility trees, including background clicks and typing, an agent pointer overlay, zooming into screen regions, and screenshots with coordinate mapping.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Computer UseOpen Safari, find the cheapest flight to Denver on Tuesday and put it in a note."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Introduction
Computer Use is an open-source MCP server — a computer-use agent (CUA) backend — that lets any coding agent use your computer the way a person does. It reads the screen through accessibility trees, clicks and types in the background so your mouse stays yours, shows an agent pointer where it is working, zooms in on small text, and drives tabs in your signed-in Chrome — on macOS, Windows and Linux.
Works with Claude Code, Codex, Cursor and MT Code, or any other MCP client, with any model — no vision model is required for interaction.
Built by Munim Technologies as the Computer Use engine of MT Code, and published here on its own.
Related MCP server: openowl
Table of contents
Quick start
Download the latest binary for your platform from Releases (
munim-computer-use-macos-universal.zip,munim-computer-use-windows-x64.zip) or build from source.Put it somewhere on your
PATH(/usr/local/bin/munim-computer-use, or%LOCALAPPDATA%\Programs\munim-computer-use\munim-computer-use.exe).macOS only: run
munim-computer-use request-permissionsonce to be prompted for Accessibility and Screen Recording.Register it with your agent:
# Claude Code — fastest: the npm launcher fetches the signed binary on first run.
# Add `--scope user` to register it for every project instead of just this one.
claude mcp add munim-computer-use -- npx -y munim-computer-use
# or point at a downloaded binary
claude mcp add munim-computer-use -- /usr/local/bin/munim-computer-use# Codex — writes the entry below into ~/.codex/config.toml for you (needs a Codex CLI
# with `codex mcp`; check with `codex mcp --help`)
codex mcp add munim-computer-use -- npx -y munim-computer-use# Codex — ~/.codex/config.toml, if you would rather edit it yourself
[mcp_servers.munim-computer-use]
command = "npx"
args = ["-y", "munim-computer-use"]# Cursor — no MCP subcommand in its CLI, so write the config. ~/.cursor/mcp.json applies
# to every project; .cursor/mcp.json in a repo applies to that one.
mkdir -p ~/.cursor && [ -s ~/.cursor/mcp.json ] || echo '{}' > ~/.cursor/mcp.json
jq '.mcpServers["munim-computer-use"] = {"command":"npx","args":["-y","munim-computer-use"]}' \
~/.cursor/mcp.json > ~/.cursor/mcp.json.tmp && mv ~/.cursor/mcp.json.tmp ~/.cursor/mcp.json// Cursor — the entry that produces, in .cursor/mcp.json
{ "mcpServers": { "munim-computer-use": { "command": "npx", "args": ["-y", "munim-computer-use"] } } }Then ask: "Open Safari, find the cheapest flight to Denver on Tuesday and put it in a note." The agent reads the UI with get_app_state, acts by element id, and verifies with screenshot.
Capability matrix
Munim Computer Use is the highlighted first column; the others are the computer-use servers people reach for. each cell comes from that project's own README or docs in September 2026 (sources under Credits and license). ✅ present · ❌ absent or not documented · ⚠️ partial.
Capability | Munim Computer Use | Codex Computer Use | OpenAI Agents API | Anthropic reference demo | Windows-MCP | MacOS-MCP | open-computer-use | computer-use-mcp (zavora) | Notes |
macOS | ✅ | ✅ | n/a | ❌ | ❌ | ✅ | ✅ | ✅ | The Anthropic demo drives a Linux desktop inside Docker, not your machine. The OpenAI Agents API (September 2026) runs a browser OpenAI hosts, so it never drives your machine either. |
Windows | ✅ | ✅ | n/a | ❌ | ✅ | ❌ | ✅ | ✅ | Windows-MCP is Windows only; MacOS-MCP is macOS only. |
Linux | ✅ | ❌ | n/a | ✅ (sandbox) | ❌ | ❌ | ✅ | ✅ | Munim Computer Use uses AT-SPI + X11; native Wayland apps get element actions but not coordinate clicks. |
Accessibility tree with element ids | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | Codex, the Agents API and the Anthropic demo are screenshot-driven. Ids let the agent press the button instead of a pixel. |
Background input (your mouse never moves) | ✅ | ✅ | n/a | n/a | ❌ | ❌ | ❌ | ❌ | Munim Computer Use addresses events to the target window: always on macOS (SkyLight and per-process events), for UI Automation patterns and classic Win32 controls on Windows; other Windows UI and all Linux pointer input (XTEST) still move the real pointer. Codex does this too, with a second cursor of its own. Every other server in this table drives the real cursor. |
Agent pointer overlay | ✅ | ✅ | ❌ | ❌ | ⚠️ | ❌ | ❌ | ❌ | Windows-MCP flashes a border around captures; Codex draws its own cursor on your screen. |
Zoom into a region at full resolution | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Anthropic's toolset has |
Screenshots carry screen-coordinate mapping | ✅ | n/a | n/a | n/a | ❌ | ❌ | ❌ | ❌ | Origin and pixels-per-point in every capture, so clicks from Retina or downscaled images land. |
Hover, wait, label query | ✅ | ⚠️ | ❌ | ⚠️ | ✅ | ⚠️ | ❌ | ⚠️ | Windows-MCP has Wait/WaitFor; MacOS-MCP has Wait; Anthropic has |
Your signed-in Chrome, own tab group or yours | ✅ | ⚠️ | ❌ | ❌ | ⚠️ | ❌ | ❌ | ❌ | Codex uses its in-app browser; Windows-MCP reads the DOM of open browsers. Munim Computer Use opens its own labelled tab group in your real Chrome, and can also take over a tab you already have open when you ask it to. |
Act and observe in one call | ✅ | n/a | n/a | ✅ | ❌ | ❌ | ❌ | ❌ |
|
Read a page's text in a signed-in tab | ✅ | ❌ | ❌ | ❌ | ⚠️ | ❌ | ❌ | ❌ |
|
Sign-in without the model seeing the password | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
Per-app and per-site allow / ask / block rules | ✅ | ⚠️ | ⚠️ | ❌ | ❌ | ❌ | ❌ | ❌ | A policy file the user controls; |
Works with any MCP client | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ | Codex Computer Use is Codex only; the Anthropic demo is Claude only. |
Identical tool surface on every platform | ✅ | n/a | n/a | n/a | n/a | n/a | ⚠️ | ✅ | 32 tools with byte-identical schemas across the Swift and Rust servers. |
Prebuilt signed binaries + npm launcher | ✅ | ✅ | n/a | ❌ | ❌ | ❌ | ✅ (npm) | ✅ (npm) | macOS universal (Developer ID signed) and Windows x64 on Releases. |
Open source | ✅ Apache-2.0 | ❌ | ❌ | ✅ | ✅ MIT | ✅ MIT | ✅ MIT | ✅ MIT |
Also looked at: mediar-ai/mcp-server-macos-use (macOS, accessibility, real input), deploymenttheory/windows-mcp-server (Windows, UIA Invoke patterns, WaitFor), nuphus-mcp (OCR + bring-your-own vision model, CDP Chrome), computer-control-mcp (PyAutoGUI + OCR), and microsoft/playwright-mcp (browser only). Corrections welcome — open an issue with a link.
Why it works well
Accessibility first, pixels second.
get_app_statereturns the app's accessibility tree with stable element ids, so the agent presses the button instead of guessing at a coordinate. It costs a fraction of the tokens of a screenshot and it is what scores highest on OSWorld-style tasks. Screenshots are for verifying and for content the tree cannot describe.What is on screen, fast.
get_app_statelists what is in view: a list or table gives its visible rows, and anything scrolled away is skipped and counted (offscreen: truelists it too). Attributes are read in one request per element. A Finder window of 2,000 files reads in 165 ms instead of 2.2 s, and a Windows list of 3,000 rows in 0.85 s instead of 2.3 s, with room left for the rest of the app.Dialogs first. While a sheet, alert or popover blocks a window, only its controls are listed, with a note saying so, so the agent answers the dialog instead of clicking behind it.
Electron and Chromium apps (macOS). Slack, VS Code, Discord and other Chromium-based apps expose their tree only to a client that asks;
get_app_stateasks, and says so while a background app is still building it.Text the tree cannot describe. When an app exposes no labelled controls, its window is read with on-device OCR (Apple Vision on macOS, Windows OCR on Windows) and the text is listed with ids that
click,hover,right_clickanddragaccept. The image never leaves the machine.ocr: "always"or"never"overrides it.Ids that fail instead of misfiring. Every id is rechecked before an action. If its element is gone, or its role or name changed (a dialog swapped its buttons, a list re-sorted), the action fails with a clear error instead of acting on whatever sits there now.
Waits as long as the app takes.
return_statewaits for the app's accessibility notifications to stop instead of a fixed pause: about 230 ms for a field edit (was 350 ms), and long enough for a sheet to finish sliding in.Background control. Events are addressed to the target window (SkyLight on macOS, UI Automation patterns and posted window messages on Windows). On macOS the agent never takes your mouse or keyboard; see Works alongside you for the exact guarantee on each platform.
Pointer overlay, not your pointer. A soft lavender agent pointer shows where the agent is acting. Your cursor is untouched.
Coordinates that land. Every screenshot and zoom carries its screen origin and pixels-per-point.
zoomcaptures any region at full physical resolution.Your browser, your logins. The Chrome extension gives the agent its own labelled tab group in your signed-in Chrome, and leaves your tabs alone unless you point it at one.
Or the tab you already have open.
browser_list_tabs all=trueshows every tab in the browser andbrowser_use_tabtakes one over in place — useful when the page is already signed in or mid-flow and re-opening the URL would throw that away. An adopted tab is not moved into the agent's group, not activated and not reloaded; cleanup releases it rather than closing it, andbrowser_release_tabhands it back early.Tabs the agent's clicks open are the agent's too. A
target=_blanklink or popup opened by an agent click or key press within 10 seconds joins that task's tabs and group, and its cleanup closes it. A tab opened from an agent tab without a recent agent action stays yours.Parallel tasks, one extension. Every MCP process gets its own tab group, and one process can run several tasks by passing a stable
session_idon its browser calls. A task cannot drive or adopt another task's tabs, and cleanup (including a process exiting) closes only its own. Any number of MCP processes share the one extension: the first owns it and the rest go through it, and if the owner exits another takes over without closing anyone's tabs. Tasks share Chrome's cookies and logins, and desktop apps and the clipboard are not isolated.Model-agnostic. No vision model is required for interaction; local models work too.
Look → act → verify.
hoverfor mouse-over menus,waitfor loads,queryto find a control by label without reading a whole tree.Act and look in one call. Pass
return_state: trueto any action (click,type_text,set_value,browser_click,browser_navigate, …) and the result carries the app's fresh accessibility tree, or the page's current snapshot. Browser actions report readiness separately: a loaded tab is not proof that delayed SPA UI has rendered. Usewait_for_selectorfor an expected visible element (see below).state_querynarrows the desktop tree the same wayquerydoes.Pages as text.
browser_readreturns a page's readable text with its headings, including what is scrolled out of view, from a tab in the background. It comes in chunks you can continue withoffset,querykeeps only the matching lines under their heading, andinclude_linkslists the links.Passwords stay out of the conversation.
browser_request_credentialsasks the user to sign in in a small Chrome window that shows the site's real origin. What they type goes straight into the page's fields and is never returned to the model.browser_snapshotnever shows a password field's value, and on the desktop, typing into password fields is refused by default.Pick up what you are looking at.
get_app_stateandscreenshottakeapp: "frontmost"for the app in front of the user, so a client can hand the agent "this window" in one step.
Works alongside you
The agent has its own pointer; yours stays yours.
macOS — guaranteed. Every action goes through accessibility (press, set value, select text, show menu, scroll bars) or through events addressed to the target app's process and window. The server never moves your pointer, never posts into the system-wide input stream, and never holds or blocks your input, so you can keep clicking and typing in other apps while the agent works — even in the same app, on another window. Events come from a private source, so a modifier you are holding does not leak into the agent's clicks. The target app may be brought forward when that is the point of the step (activate_app, or handing you a password field), but not on every action.
Command shortcuts sent to a background app run through its menu bar (macOS gives menu key equivalents only to the front app), so cmd+W, cmd+S and the like work without bringing it forward. A minimized window, or a hidden app, is moved for the moment it needs pixels or input (typing, a coordinate click, a screenshot, OCR) onto an invisible display, where it works while nobody sees it; it is minimized or hidden again after 20 idle seconds, when another app's window needs the display, or when the server exits. If you bring the app up yourself meanwhile, its windows come back on screen where they were. Accessibility actions on such a window (press, set value) need none of this. COMPUTER_USE_PARK_WINDOWS=0 turns it off, and those actions then return an error asking for the window to be restored.
The exceptions refuse instead of borrowing your pointer: a click, hover or scroll with no target app (coordinates over the desktop before any get_app_state), and drags that leave the source window (between apps, or onto the desktop). The error says what to pass instead.
Windows — best effort, reported. Element presses (Invoke), set_value, select_text and scrolling through UI Automation's ScrollPattern never touch your pointer, and classic Win32 controls also take clicks, hovers, wheel and drags as posted window messages. Other UI (Chromium, Electron, WPF, UWP) only reacts to real mouse input, so those clicks, hovers and drags move your pointer, and type_text/press_key go to the focused window. Whenever the real pointer was used, the result says via cursor. A coordinate action on an element that another app's window covers is refused rather than clicking that window, and pointer and keyboard actions on a minimized window return an error asking for it to be restored (accessibility actions still work on it).
Linux — pointer actions use it. XTEST input moves the real pointer and goes to the focused window, and results say via cursor. Element actions through AT-SPI (press, set value, insert text, select) do not move it.
Apps and sites the agent may use
You decide which apps and websites the agent may touch. Put a policy.json in the server's support directory (~/Library/Application Support/computer-use on macOS, %LOCALAPPDATA%\munim-computer-use on Windows, ~/.local/share/munim-computer-use on Linux; munim-computer-use identity prints it as supportDir), or point COMPUTER_USE_POLICY at a file anywhere. An app that embeds the server under its own profile reads the file from its own support directory:
{
"apps": { "Keychain Access": "block", "com.apple.MobileSMS": "ask" },
"sites": { "bank.example": "block", "mail.google.com": "ask", "*": "allow" }
}allow,askorblockper app or site.askshows the user a prompt the first time the agent reaches for it (a system dialog for apps, a small Chrome window for sites), and an approval lasts until that agent task ends. An unanswered prompt counts as no after two minutes.Apps match their name or bundle id, ignoring case. The rule applies when the agent reads, captures or activates the app. Actions then target elements from a snapshot that was allowed.
Sites match a host and all its subdomains, and the most specific pattern wins. They are checked on every browser action against the page the tab is showing at that moment, so following a link into a blocked site does not get around it.
*sets the default, which is otherwiseallow, so{"apps": {"*": "block", "Notes": "allow"}}is an allow-list.Changes apply at once, with no restart. A file that is not valid blocks everything rather than being ignored, so a typo cannot quietly turn a block into an allow.
This is a guard rail for an agent that follows its instructions, not a sandbox. A coordinate click lands wherever it points, and a whole-display screenshot shows every window.
Tools (32)
Area | Tools |
See |
|
Act |
|
Clipboard |
|
Browser |
|
Every action tool also takes return_state, which returns current state after the action in the same call.
Browser readiness and safe re-observation
browser_click, browser_type, browser_press_key, and browser_navigate accept
wait_for_selector together with return_state: true. browser_snapshot accepts
the same wait without performing an action:
{"tab_id": 42, "index": 0, "return_state": true,
"wait_for_selector": "[role=dialog] button", "wait_timeout_ms": 3000}The extension polls for any matching visible element in the top-level document,
independently of resource loading, including same-document rendering. It shares
the snapshot's visibility rules: at least 2×2 CSS pixels; no hidden, aria-hidden,
inert or fully transparent ancestor. Self-transparent native form controls remain
available for styled upload/checkbox hit targets; transparent containers do not.
This is not a guarantee of clickability or
lack of occlusion. Frames and shadow
roots are not searched. Choose a selector specific to the new expected UI;
an element already present can satisfy it immediately. No arbitrary page script
is accepted, and a selector may not test the value attribute: React mirrors a
controlled input's value there, password fields included, so a wait could
otherwise confirm a password one guessed character at a time.
wait_timeout_ms: defaults to8000, accepts integers0–10000;0checks once. Requireswait_for_selector. The budget covers polling, not browser/API latency or user site-approval prompts. Site rules are rechecked before every observation and before the final snapshot.readiness: met (visible_selector, … ms): the specified condition was observed, not a guarantee that all network requests, animations, or application work ended.readiness: timeout: the condition was not observed within the budget. The action's success remains separate, and the current snapshot is still returned.readiness: error: the condition could not be evaluated (for example invalid CSS); the action can still have succeeded. Correct the wait and re-observe.Without a selector,
return_stateretains the short pause and bounded navigation-load wait, but reportsnot_requested (document_load, … ms)rather than claiming SPA readiness. A navigation-load budget expiry reportstimeout. A plainbrowser_snapshotwithout a wait remains an immediate observation.
If a returned snapshot looks unchanged, do not repeat a non-idempotent click just because expected UI is absent. Wait/re-observe first:
{"tab_id": 42, "wait_for_selector": "[role=dialog] button", "wait_timeout_ms": 3000}Send this to browser_snapshot, inspect its readiness and fresh indices, then
choose the next action. A timeout or tool error does not roll back a click,
submission, typing, or navigation. Use current state to decide whether a retry is
safe. Invalid wait types/bounds or an action wait without return_state: true are
rejected before acting. null for an optional argument means the same as leaving
it out. These options require an updated extension as well as the native server;
when an older extension ignores them, the result says so and asks for a reload.
Regression checks: node --test chrome-extension/background.test.mjs covers the
wait contract and site-policy guards. The real-browser CI harness
scripts/e2e-browser.mjs tests a dialog delayed by 1100 ms without navigation,
one click only, plus timeout and read-only recovery. The dependency-free CDP
regressions cover both snapshot scoping/pagination and SPA readiness. Run against
an isolated Chrome/Chromium profile; no Playwright module is required:
node scripts/snapshot-dom.test.mjs --chrome /path/to/chrome
# Extract the pre-fix source; each suite must fail independently:
git show deb4c7a:chrome-extension/background.js > original-background.js
node scripts/snapshot-dom.test.mjs --chrome /path/to/chrome --source original-background.js --suite snapshot
node scripts/snapshot-dom.test.mjs --chrome /path/to/chrome --source original-background.js --suite readinessbrowser_snapshot automatically scopes to the topmost visible dialog when one is
open, so covered background controls cannot consume the dialog's budget. A
dialog's portalled popups stay in scope: elements its controls point at through
aria-controls/aria-owns, open popovers, and controls painted on top of the
dialog, such as a select's options rendered at the end of <body>. Hidden
and inert subtrees are excluded; reachable offscreen controls remain available.
Native modeless dialogs and explicit aria-modal="false" do not scope the page.
Native modal dialogs and explicit aria-modal="true" are definitive candidates.
Implicit ARIA dialogs are heuristically scoped only when they cover the viewport
centre and are topmost there, rather than merely at their own centre. With no
hit-test stack, centre coverage is the fallback. Corner chat widgets and bottom
cookie banners therefore retain the page's indexed controls.
It returns up to limit controls (default and maximum 250), reports scope,
offset, limit, and total, and explicitly says when it is truncated and which
offset to request next. Continue on the same tab_id and session_id. Indices
are global within the selected scope and stay actionable across pages of an
unchanged UI; after a UI change or navigation, take a fresh snapshot from
offset=0. An action's return_state also starts at offset=0 and uses the same
scope and truncation reporting.
Names, argument shapes and descriptions are identical on every platform, and CI enforces it (node scripts/check-tool-parity.mjs); a model that learned them on a Mac needs nothing new on Windows. To change a tool, edit both literals at once with scripts/tool-defs.mjs rather than by hand.
Repository layout
Directory | What | Build |
| Swift server on the Accessibility API and ScreenCaptureKit (macOS 14+) |
|
| Rust server: UI Automation on Windows, AT-SPI + X11 on Linux |
|
| Chrome extension + native messaging host for the | Load unpacked; |
| Headless Linux build of the Rust server for registry introspection (Glama and similar); no desktop control inside a container |
|
Build from source
On Fedora, install the native development libraries before building the Rust server (in addition to your Rust toolchain):
sudo dnf install gcc pkgconf-pkg-config clang-devel pipewire-devel \
libdrm-devel mesa-libgbm-devel mesa-libEGL-devel wayland-devel libxcb-develThese are needed by the screen-capture dependencies even on X11. In particular,
a missing libpipewire-0.3.pc during the build means pipewire-devel is missing.
# macOS
cd macos && swift build -c release
# Windows / Linux
cd windows-linux && cargo build --releaseLinux notes: element actions work everywhere; coordinate clicks need an X11 or XWayland client, since native Wayland apps do not expose absolute geometry.
Tests
cd windows-linux && cargo test # Rust server
node chrome-extension/background.test.mjs # extension, against a fake Chrome
node scripts/check-tool-parity.mjs # Swift and Rust tool lists match
# Snapshot and SPA readiness regressions (dependency-free CDP, isolated profile):
node scripts/snapshot-dom.test.mjs --chrome <Chrome for Testing binary>
# The browser tools end to end, in a throwaway Chrome for Testing profile that
# never touches your own Chrome (npx playwright install chromium to get one):
node scripts/e2e-browser.mjs --server <munim-computer-use binary> --chrome <Chrome for Testing binary>Chrome extension (optional)
chrome://extensions→ Developer mode → Load unpacked → selectchrome-extension/.Register the native messaging host:
munim-computer-use install-native-host(for examplenpx -y munim-computer-use install-native-host), or from a checkoutsh chrome-extension/install.sh(macOS/Linux) /powershell -File chrome-extension/install.ps1(Windows), which find the build and run the same command. PointCOMPUTER_USE_PATHat the binary if it is not in the default build location.
If the extension still gets Specified native messaging host not found., or the browser tools say the extension is not connected, run munim-computer-use check-native-host in an ordinary terminal. It looks the host up the way Chrome does (on Windows: registry views and policies; everywhere: the manifest's encoding, name, type and allowed origins), starts it with Chrome's command line, and names the step that would fail. Two Windows causes look fine from the inside. An installer run by an MSIX-packaged app (some agent desktop apps are packaged) writes to a private, virtualised copy of HKCU and %LOCALAPPDATA% that Chrome never reads. One run by a sandbox with its own Windows account writes to that account's HKCU. The check detects both; run install-native-host again from your own terminal. install-native-host reports the same findings under problems.
The standalone server's host is com.munimtech.computer_use.desktop; MT Code, which bundles this server, registers com.munim.mtcode.desktop. The extension connects to every host it knows at once and answers each on its own connection, so MT Code and a standalone server (npx, Claude Code, Cursor, a checkout) can both drive Chrome at the same time, each in its own tab groups.
Sandboxed agents (Windows)
Some agents run their commands in a sandbox: a restricted or low-integrity token, a separate Windows account, or another desktop. A server started there can still list windows and their titles, but other apps' UI Automation refuses it and captures come back black, without any error. When that happens the server says so: get_app_state explains an empty tree, screenshot and zoom flag a capture that is one flat colour, and activate_app checks that the window really came forward. Each names the cause it found (for example "runs at low integrity" or "runs as another account"). Retrying from the same context will not help; configure the agent to start the MCP server in your own, unsandboxed context.
Environment flags
Variable | Effect |
| Hide the |
| Do not draw the agent pointer |
| How long the pointer stays after the last tool call (default 8) |
| Allow typing into password fields (refused by default) |
| Remote-desktop mode: input takes over the real pointer |
| macOS: do not move minimized or hidden windows to an invisible display to act on them |
| Where to read the app and site policy (default |
Remote control
Normally this server never touches the pointer: coordinate clicks are routed to a specific window, keystrokes are posted to a specific process, and an action that cannot be targeted is refused rather than taking over the machine. That is what lets an agent work while the user keeps using their computer.
COMPUTER_USE_REMOTE_CONTROL=1 inverts that contract for one process, for the
case where a person is watching this machine's screen from another one and is
steering it themselves. Then click, right_click, drag, hover and
scroll move the real cursor and type_text and press_key go to whatever is
focused, the way Chrome Remote Desktop or Screen Sharing behave. scroll also
accepts x/y so the wheel acts over the point the viewer scrolled at, and the
agent-cursor overlay stays hidden — there is only one pointer now.
A host that runs both an agent and a viewer runs them as two processes, so turning this on for the viewer never takes the pointer away from the user on the agent's behalf.
Embedding in an app
An app can ship this binary inside its own bundle and run it under its own identity, so it never shares a browser bridge, agent-cursor app or native-messaging host with a standalone install on the same machine. MT Code does exactly this. Pass a profile, either as a JSON object or a path to a JSON file, with --profile <json|file> (any position) or COMPUTER_USE_PROFILE:
{
"name": "example-desktop",
"envPrefix": "EXAMPLE_DESKTOP_",
"agentCursorName": "ExampleAgentCursor",
"agentCursorBundleId": "com.example.agent-cursor",
"nativeHostNames": ["com.example.desktop"],
"extensionIds": ["abcdefghijklmnopabcdefghijklmnop"],
"nativeHostDescription": "Example desktop control bridge"
}Key | Default | Meaning |
| none (standalone paths) | Moves the support dir, bridge socket and Windows pipe under this name |
|
| Native-host wrapper, profile copy; on macOS also the bridge socket and a bare build's overlay app |
|
| Where the MCP server and the Chrome relay meet |
| none | Tunables are read as |
|
| The pointer overlay's app, executable and window-class name, and its macOS bundle id |
| none | Default |
|
| What |
Each of supportDir, bridgeSocket, envPrefix, agentCursorName, agentCursorBundleId and historyDir can also be overridden by COMPUTER_USE_<SNAKE_CASE> (for example COMPUTER_USE_SUPPORT_DIR), which wins over the profile. munim-computer-use identity prints the resolved values.
Browser bridge. Run
munim-computer-use install-native-hostwith the same profile. It writes a wrapper that relays Chrome into this identity's bridge (replaying the profile), and a host manifest for each name in every Chrome/Chromium profile directory (the registry on Windows). It rewrites nothing that is already current, so an app can call it on every launch.Extension. Use the stock extension (it connects to
com.munim.mtcode.desktopandcom.munimtech.computer_use.desktop), or build a variant with its own host names, tab-group title and key:node scripts/build-extension.mjs --out <dir> --host com.example.desktop --group-title "Example" --key <base64>. It prints the variant's extension id forextensionIds.macOS permissions. The MCP server is a bare executable, so Accessibility and Screen Recording are granted to the app that spawns it. Only the agent-cursor overlay has a bundle of its own; ship
<agentCursorName>.app(a copy of the binary plus anLSUIElementInfo.plist) beside the binary, or it is materialised undersupportDiron first use.
Prompting your agent
Look → act → verify. get_app_state for ids, act by id with return_state: true to see the result in the same call, and screenshot when the tree cannot show it. Read pages with browser_read, and let the user type passwords through browser_request_credentials. Use zoom for small text, hover for menus that appear on mouse-over, wait after loads, keyboard shortcuts for stubborn widgets. The system-prompt text MT Code gives its agents lives in CodexDeveloperInstructions.ts and is a good starting point.
Contributing
The server is developed here. MT Code ships it by pinning a release of this repository (native/munim-computer-use.json), so changes reach MT Code with the next release. Issues and pull requests are welcome.
Credits and license
Designed and built by Munim Technologies (Munim, Inc.) for MT Code. On-screen pruning, dialog scoping, notification-based settling, OCR fallback, menu shortcuts and window parking follow arc-cua (MIT). Copyright 2026 Munim, Inc. Licensed under the Apache License 2.0; see LICENSE.
Comparison sources: Codex Computer Use and its docs · OpenAI Agents API computer use · Anthropic computer-use demo · CursorTouch/Windows-MCP · CursorTouch/MacOS-MCP · QwenLM/open-computer-use · zavora-ai/computer-use-mcp · mediar-ai/mcp-server-macos-use · deploymenttheory/windows-mcp-server · nuphus-mcp · computer-control-mcp · microsoft/playwright-mcp
Available Tools
32 toolsactivate_appActivate appAIdempotent
Bring an app's windows to the foreground and give it keyboard focus. type_text, press_key and element-id actions such as click and set_value do not need it; use it only when the app itself must be in front. Side effect: the window the user was working in loses focus.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name, bundle id, or pid exactly as reported by list_apps | |
| window_id | No | Windows only: bring this one window forward instead of the app's frontmost window, for an app with several windows such as Chrome. Use the window_id from a get_app_state window line; it must belong to app. The result names the window in front afterwards. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare non-readOnly, idempotent and non-destructive, but do not convey the user-visible focus steal. The description adds that context ('the window the user was working in loses focus'), which is exactly the behavioral trait annotations miss. It stops short of describing what happens if the app isn't running, but the key side effect is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, front-loaded with the action, followed by the usage condition and the side effect. No filler or repetition of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers purpose, exclusions, and the main side effect for a small two-parameter tool whose schema is fully documented. Minor gaps: nothing about failure when the app is not running or how activation interacts with multiple displays, but the essentials for correct invocation are present.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, including the app identifier format and the Windows-only window_id semantics, so the schema carries the parameter burden. The description adds no parameter detail beyond that, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Bring an app's windows to the foreground and give it keyboard focus') with the observable effect, and distinguishes itself from the element-id action siblings (click, set_value, type_text, press_key). An agent can tell exactly what this does versus the other input tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says the tool is NOT needed for type_text, press_key and element-id actions, and gives the positive selection condition: 'use it only when the app itself must be in front.' That is a clear when/when-not with the competing tools named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_clickClick in browserADestructive
Click in one of the agent's tabs, either an element by its index from browser_snapshot (preferred) or a point given in page coordinates. Pass index or x and y, not both. Works on a background tab. A tab the click opens (a target=_blank link or a popup) becomes one of the agent's tabs; find it with browser_list_tabs. Use click for native app windows. A click can submit forms or follow links, so snapshot first.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Page x coordinate in CSS pixels, used together with y when no index is given | |
| y | No | Page y coordinate in CSS pixels, used together with x when no index is given | |
| index | No | Element index from the latest browser_snapshot of this tab. Preferred over coordinates. | |
| tab_id | Yes | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. | |
| return_state | No | Append a current snapshot and readiness status after acting. Without wait_for_selector, only a short pause and bounded navigation wait are used; SPA readiness is not guaranteed. Snapshot indices replace earlier ones. A timeout does not mean the action failed: re-observe with browser_snapshot before repeating it. | |
| wait_timeout_ms | No | Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action. | |
| wait_for_selector | No | Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare the destructive/non-idempotent/open-world profile, and the description adds genuinely new behavior beyond them: the click works on a background tab, and a click that opens a target=_blank link or popup spawns a new tab discoverable via browser_list_tabs. It does not restate the safety hints, and return_state/wait semantics are left to the schema, so this is strong but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the primary locator contract, then the tab/popup behavior, then the caution about side effects. Every sentence carries information, though the description is fairly dense for what is fundamentally a two-mode locator tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an 8-parameter, no-output-schema action tool this covers the agent-relevant behavior: tab targeting, background-tab support, new-tab side effects, and the snapshot prerequisite. The return_state, wait_for_selector and wait_timeout_ms semantics are handled thoroughly in the schema, so leaving them out of the description is reasonable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, and the description earns an increment by stating the mutual-exclusion rule ('Pass index or x and y, not both') and the preference for index over coordinates, which the property-level descriptions only hint at.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('click') and resource (an element by index from browser_snapshot, or a page coordinate point), and explicitly separates itself from the sibling 'click' tool, which handles native app windows. An agent can distinguish it from right_click, hover, and drag without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing rules: use browser_snapshot first, pass index OR x/y but never both, use click for native windows, and find popups via browser_list_tabs. The 'snapshot first' warning also explains the precondition (clicks can submit forms or follow links), which is exactly the when-to-use context an agent needs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_close_all_tabsClose all agent tabsADestructiveIdempotent
Close every tab the agent opened, including tabs its clicks opened, and remove its tab group. Tabs taken over with browser_use_tab are released back to the user, not closed. Call this when finished with the browser so no empty group is left in the user's tab strip. The MCP process also runs this automatically when the Computer Use session ends. Unsaved state in the agent's tabs is lost.
| Name | Required | Description | Default |
|---|---|---|---|
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations (destructiveHint/idempotentHint): it discloses what is destroyed ('Unsaved state in the agent's tabs is lost'), that taken-over tabs are released rather than closed, and that the MCP process runs the same cleanup automatically at Computer Use session end. That last point materially affects whether the agent should call it at all.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, front-loaded with the action and each carrying distinct information: scope, the use_tab exception, the recommended call timing, the automatic behavior, and the data-loss warning. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and a single schema-documented parameter, the description covers everything an agent needs: what is closed, what is not, when to call it, what is lost, and that cleanup may happen automatically. Nothing material is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so session_id is fully documented in the schema; the description adds nothing about the parameter (e.g. how session scoping interacts with the auto-cleanup at session end). Baseline 3 applies when the schema carries the parameter burden alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Close every tab the agent opened') and precisely bounds scope: agent-opened tabs including those opened by clicks, plus removal of the tab group. This is clearly distinguishable from the single-tab sibling browser_close_tab and from browser_release_tab.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit trigger ('Call this when finished with the browser') and explains the consequence of not calling it (leftover empty group in the user's tab strip). It also notes the agent does not need to close tabs it merely took over with browser_use_tab, but it never names browser_close_tab as the alternative for closing a single tab.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_close_tabClose browser tabADestructiveIdempotent
Close one of the agent's tabs, discarding any unsaved page state. A tab taken over with browser_use_tab is released rather than closed — it belongs to the user. Use browser_close_all_tabs to clean up everything at the end of a task.
| Name | Required | Description | Default |
|---|---|---|---|
| index | No | 1-based position within the agent's tabs; fallback mode only, when tab_id is unavailable | |
| tab_id | No | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds important behavioral context beyond the annotations: it discloses that closing a tab discards unsaved page state, and that tabs taken over with browser_use_tab are released rather than closed because they belong to the user. The annotations already declare destructiveHint=true and idempotentHint=true, and the description aligns with these. It doesn't contradict the annotations, and it adds meaningful context about the tool's side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences with zero waste. It front-loads the core action and side effect, then provides the sibling distinction and the special case. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a destructive tab-closing tool. It covers the core behavior, the side effect (discarding unsaved state), the special case for user-owned tabs, and the alternative for bulk cleanup. The only minor gap is that it doesn't explicitly state what happens if both index and tab_id are provided, but the schema's 'fallback mode only' note covers this. With no output schema, the description doesn't need to explain return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents all three parameters (index, tab_id, session_id). The description doesn't add much beyond what the schema provides, but it does clarify the fallback relationship between index and tab_id ('fallback mode only, when tab_id is unavailable'). This is a baseline 3 because the schema carries the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: closing one of the agent's tabs and discarding unsaved page state. It also distinguishes itself from the sibling tool browser_close_all_tabs, which is for cleaning up everything at the end of a task. The verb 'close' and resource 'tab' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool versus alternatives: use browser_close_all_tabs to clean up everything at the end of a task. It also clarifies the special case of tabs taken over with browser_use_tab, which should be released rather than closed. This is clear, actionable guidance for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_list_tabsList browser tabsARead-onlyIdempotent
List the tabs in the agent's own Chrome tab group, marking the active one, with the tab_id each other browser tool needs. Pass all=true to see every tab open in the browser, including the user's, so you can pick one to drive with browser_use_tab. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| all | No | List every tab in the browser, not just the agent's (default false) | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description reinforces 'Read-only'. It adds meaningful behavioral context beyond annotations: the agent's own tab group is the default scope, all=true expands to every tab, the active tab is marked, and the returned tab_id is needed by other browser tools.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with no filler. The core behavior is front-loaded, the all=true variant is explained exactly where needed, and the safety note is appended last. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only listing tool with no output schema, the description adequately conveys the key returned data (tab list, active marker, tab_id) and the parameter behavior that matters for invocation. It could be slightly richer about the exact output shape, but nothing critical is missing for selecting and calling the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents both parameters. The description adds prose meaning for all ('Pass all=true to see every tab...') but does not discuss session_id beyond what the schema provides. This matches the baseline for fully documented parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'List the tabs in the agent's own Chrome tab group', and clarifies the output includes active marking and tab_id. This clearly distinguishes it from sibling tools like browser_snapshot or browser_select_tab while positioning it as the listing companion to browser_use_tab.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete guidance on when to use all=true (to see the user's tabs and pick one to drive with browser_use_tab) and clarifies the default scope. It does not explicitly state when not to use the tool, but the context is clear enough for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_open_tabOpen browser tabA
Open a URL in a new background tab inside the agent's own labelled tab group in the user's signed-in Chrome, and return its tab_id for browser_snapshot, browser_click, browser_type and browser_navigate. The tab opens in the background, so the user's browsing is not interrupted. Use browser_use_tab instead when the user already has the page open and signed in. Requires the Computer Use Chrome extension; a limited fallback mode applies without it.
| Name | Required | Description | Default |
|---|---|---|---|
| url | No | Absolute URL to open (default about:blank) | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses that the tab opens in the background, that it lives in the agent's own tab group, that it returns a tab_id for other browser tools, and that it requires the Computer Use Chrome extension with a limited fallback mode. These are material behavioral traits not present in the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each earning its place: the core action and return value, the background behavior, and the alternative tool with extension requirement. The most important operational detail is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter optional tool with no output schema, the description is complete enough: it states what the tool returns, how the tab behaves, when to use a sibling instead, and the extension prerequisite. The agent has what it needs to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema fully documents url and session_id. The description adds context about the returned tab_id and its downstream use, but does not add parameter-level semantics beyond the schema. This matches the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource ('Open a URL in a new background tab'), clarifies the tool's distinguishing behavior (background tab, agent's own labelled tab group), and names the return value (tab_id). This clearly differentiates it from sibling tools like browser_navigate and browser_use_tab.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool: to open a URL in a new background tab without interrupting the user. It also names the alternative and the exact condition for choosing it: 'Use browser_use_tab instead when the user already has the page open and signed in.' This is direct usage guidance, not just implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_press_keyPress key in browserADestructive
Press Enter, Tab, Escape or Backspace in one of the agent's tabs, for example Enter to submit a form after browser_type. Only these four keys are supported; use browser_type for characters. Tab moves focus to the next field and says which element has it; it fails rather than leave focus where it was. Enter can submit forms and Backspace deletes, so check the page state with browser_snapshot first.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | Key to press | |
| tab_id | Yes | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. | |
| return_state | No | Append a current snapshot and readiness status after acting. Without wait_for_selector, only a short pause and bounded navigation wait are used; SPA readiness is not guaranteed. Snapshot indices replace earlier ones. A timeout does not mean the action failed: re-observe with browser_snapshot before repeating it. | |
| wait_timeout_ms | No | Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action. | |
| wait_for_selector | No | Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already flag destructiveHint=true and non-idempotent, but the description explains the concrete consequences (Enter can submit forms, Backspace deletes) and discloses a non-obvious behavior not derivable from any structured field: Tab fails rather than silently leaving focus where it was. It also advises snapshotting first, which is real operational guidance.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the verb, resource and key set, and every subsequent sentence carries distinct information (key limitation, Tab behavior, destructive consequences). It is four dense sentences rather than one, so it is efficient but not maximally tight.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still covers the destructive risk, the accepted inputs, and the recommended pre-flight check, while the schema handles wait/return-state mechanics. Nothing an agent needs to call this safely is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description still adds value by scoping the key enum to exactly four values and clarifying that any other character intent belongs to browser_type. It does not restate the meaning of return_state, wait_for_selector, or tab_id, which the schema already documents.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (press) and resource (one of four named keys in a tab), and immediately distinguishes itself from browser_type by routing character input there. An agent can tell exactly what this tool does and does not do without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use (Enter to submit a form after browser_type), an explicit limitation (only these four keys; use browser_type for characters), and a prerequisite (check page state with browser_snapshot first). Alternatives and conditions are named, not inferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_readRead page textARead-onlyIdempotent
Read the text of the page in one of the agent's tabs: headings marked with #, then paragraphs, lists and table text in reading order, including content scrolled out of view. Use it to read an article, results or documentation; use browser_snapshot to find something to click. Works on a background tab. Long pages come back in chunks: the result says where it stopped, and offset continues from there. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| query | No | Only return lines containing this text (case-insensitive), each with its neighbouring lines and the heading above it | |
| offset | No | Character offset to start from, to continue a page that was cut off (default 0) | |
| tab_id | Yes | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| max_chars | No | Characters to return (default 20000, at most 200000) | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. | |
| include_links | No | Also list the page's links as text and URL (up to 150) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only/idempotent/open-world safety, but the description adds genuine behavioral context beyond them: it works on a background tab, includes scrolled-out content, and returns long pages in chunks with a stated stop point. The trailing 'Read-only' merely restates readOnlyHint, keeping this short of a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads what is read, then format, then usage/alternatives, then background-tab and chunking behavior. Every clause earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a full-coverage, fully-annotated read tool with no output schema, the description covers purpose, alternative routing, return shape (chunking + offset continuation) and safe concurrent use (background tab). Nothing needed to call it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and all six parameters are documented in-schema, so the schema does the heavy lifting. The description only reinforces offset-based continuation, adding no syntax or semantics beyond what the schema already states, which is the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (read) plus resource (the page text in one of the agent's tabs) and even the output format (headings with #, paragraphs, lists, table text in reading order). It explicitly distinguishes itself from browser_snapshot, so an agent can route without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use cases (read an article, results or documentation) and names the alternative with its own use case (browser_snapshot to find something to click). The selection condition between the two read-oriented siblings is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_release_tabRelease adopted tabAIdempotent
Hand a tab taken over with browser_use_tab back to the user: the agent stops driving it and the page is left exactly as it is. Call it as soon as you are done with an adopted tab. Use browser_close_tab for tabs the agent opened itself.
| Name | Required | Description | Default |
|---|---|---|---|
| tab_id | Yes | tab_id of a tab previously adopted with browser_use_tab | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds meaningful behavioral context beyond the annotations: the agent stops driving the tab and the page remains unchanged. This clarifies that the operation releases control rather than closing or modifying the page. It does not describe error/ownership edge cases, but the core behavioral contract is well disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three concise sentences with no filler: the first defines the action, the second states when to call it, and the third points to the relevant alternative. Key information is front-loaded and every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool with full schema coverage and helpful annotations, the description is complete enough. It conveys purpose, timing, behavior, and the key alternative without requiring an output schema or further elaboration.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and both parameters are well documented in the schema. The description reinforces that tab_id must refer to a previously adopted tab but does not add new semantic details beyond that. This meets the baseline for well-covered parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: it hands an adopted tab back to the user and stops agent control. It also clearly distinguishes from browser_close_tab by stating the page is left exactly as it is, so the agent can tell this tool apart from its sibling.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives an explicit timing rule: 'Call it as soon as you are done with an adopted tab.' It also names the alternative, browser_close_tab, for tabs the agent opened itself, making when-to-use and when-not-to-use clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_request_credentialsAsk the user to sign inA
Ask the user to sign in on the page in one of the agent's tabs without the values passing through you. Chrome shows the user a small window naming the site's real origin, with one field per input you list; what they type goes straight into those inputs and is never returned to you. Use it for passwords, verification codes and any secret instead of browser_type, and never ask the user to paste a secret into the conversation. Take the input indices from browser_snapshot (password inputs show as input[password]), then click the page's sign-in button yourself. Waits until the user answers, cancels, or the time runs out.
| Name | Required | Description | Default |
|---|---|---|---|
| fields | Yes | The inputs to fill, in the order the user should see them (1 to 6) | |
| reason | No | One sentence the user sees explaining why you need to sign in | |
| tab_id | Yes | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. | |
| timeout_seconds | No | How long to wait for the user (default 180, at most 600) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds substantial behavior beyond the annotations: the values never return to the agent, Chrome names the site's real origin in its own window, one field is shown per listed input, and the call blocks until the user answers, cancels, or times out. None of that is derivable from readOnlyHint/openWorldHint/idempotentHint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense but front-loaded with purpose and the safety rule; every sentence carries an instruction the agent needs. It runs a bit long for the number of concepts, but there is little pure filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers the privacy model, the input-source workflow, the blocking behavior, and the tool's relationship to browser_type. It stops short of saying what the call yields on cancel or timeout, which matters since there is no output schema, but the coverage is otherwise strong.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline would be 3, but the description adds real semantics: fields are shown in the order given, indices come from the latest browser_snapshot, and password inputs surface as input[password]. It does not explain the session_id or timeout_seconds behavior beyond the schema, which is fine.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Ask the user to sign in on the page in one of the agent's tabs') and explicitly distinguishes itself from the closest sibling: use this 'instead of browser_type' for secrets. An agent can route between the two without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use (passwords, verification codes, any secret), explicit when-not (never ask the user to paste a secret into the conversation), and names the alternative (browser_type). It also supplies the surrounding workflow: pull indices from browser_snapshot, then click the sign-in button yourself.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_select_tabSelect browser tabAIdempotent
Make one of the agent's tabs the visible one in its window, for example before capturing it with screenshot. browser_snapshot, browser_click and browser_type work on background tabs, so most tasks never need this. The agent's group lives in the user's Chrome window, so this changes which tab that window shows; use it sparingly. The user's own tabs are never selected.
| Name | Required | Description | Default |
|---|---|---|---|
| index | No | 1-based position within the agent's tabs; fallback mode only, when tab_id is unavailable | |
| tab_id | No | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses the real-world side effect: the agent's tab group lives in the user's Chrome window, so selecting a tab changes what the user sees. It also clarifies that the user's own tabs are never selectedholistically and that the action is idempotent, matching idempotentHint=true.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences with no filler. It front-loads the core purpose, then adds usage guidance and side-effect warnings, with every sentence earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tab-selection tool, the description covers purpose, when to use it, when not to use it, side effects, and scope boundaries. The input schema fully documents all parametersable, and there is no output schema, so no return-value explanation is required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add parameter-level meaning beyond what the schema already provides, but it does reinforce the fallback relationship for index and the source of tab_id indirectly through the tool description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Make one of the agent's tabs the visible one in its window.' It also differentiates from related browser tools by noting that browser_snapshot, browser_click, and browser_type work on background tabs, clarifying why this tool is rarely needed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly gives a usage example ('before capturing it with screenshot'), explains that most tasks never need it because other browser actions work on background tabs, and warns to use it sparingly because it changes the user's Chrome window. This is strong when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_snapshotSnapshot page elementsARead-onlyIdempotent
List the interactive elements (links, buttons, inputs) on the page in one of the agent's tabs, with the index each one has for browser_click, plus the page title and URL. Inputs show their type, such as input[password]. Works on a background tab, so the user can be looking at something else. Use it before every browser_click, because indices change when the page changes; use browser_read for the page's text. Read-only. Automatically scopes to the topmost visible dialog when one is open, excluding hidden/inert controls but retaining reachable offscreen controls. Results report scope, total and truncation. Continue with offset and limit on the same tab/session; indices stay global across pages of an unchanged UI. Snapshot again after UI changes.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Maximum controls returned per page (default 250, maximum 250). | |
| offset | No | Control offset for pagination (default 0). Use the continuation offset from the previous result on an unchanged UI. | |
| tab_id | Yes | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. | |
| wait_timeout_ms | No | Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action. | |
| wait_for_selector | No | Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the readOnly/idempotent annotations, the description discloses that it works on a background tab, auto-scopes to the topmost visible dialog, excludes hidden/inert controls while retaining reachable offscreen ones, and reports scope, total and truncation. These are non-obvious behavioral traits an agent needs and that annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core purpose and the browser_click relationship in the first sentence, then layers scope/pagination details. Dense but every sentence carries distinct information; the only cost is a slightly long single block without visual separation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter tool with no output schema, the description covers what is returned (elements, indices, title, URL, scope/total/truncation), pagination continuation, dialog scoping, and staleness. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline would be 3, but the description adds real meaning: offset/limit continue pagination on the same tab/session and 'indices stay global across pages of an unchanged UI', which clarifies how the paging parameters interoperate. It adds value on top of a complete schema without fully re-documenting every parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List the interactive elements ... on the page'), names the artifact it produces (indices for browser_click, page title and URL), and explicitly differentiates from the sibling browser_read ('use browser_read for the page's text'). An agent can distinguish it from all nearby tools without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use ('Use it before every browser_click, because indices change when the page changes'), names the alternative for a different need (browser_read for text), and restates the trigger ('Snapshot again after UI changes'). This is exactly the when/when-not/alternative guidance at the top of the rubric.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_typeType in browserADestructive
Type text into the field that currently has focus in one of the agent's tabs; browser_click the field first. Text is inserted at the caret without clearing existing content. Use browser_press_key for Enter, Tab, Escape or Backspace, and type_text for native apps.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Exact text to type into the focused field | |
| tab_id | Yes | tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. | |
| return_state | No | Append a current snapshot and readiness status after acting. Without wait_for_selector, only a short pause and bounded navigation wait are used; SPA readiness is not guaranteed. Snapshot indices replace earlier ones. A timeout does not mean the action failed: re-observe with browser_snapshot before repeating it. | |
| wait_timeout_ms | No | Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action. | |
| wait_for_selector | No | Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=false, so safety is covered externally. The description adds real behavioral context the annotations don't: text is inserted at the caret and does not clear existing content, which is exactly what an agent needs to predict the effect. It doesn't cover return_state/snapshot side effects, but those are documented in the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the action and precondition, then the no-clearing behavior, then the alternatives. Every clause carries information; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool annotated for safety and 100% schema coverage, the description covers the precondition, the insertion semantics, and sibling routing. Minor gap: it doesn't warn about the potential destructive nature that destructiveHint implies, but the schema's return_state text handles re-observation guidance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description reinforces tab_id context ('one of the agent's tabs') and focus behavior but adds no syntax, formatting, or edge-case detail beyond what the schema already provides per parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource: type text into the currently focused field in one of the agent's tabs. It explicitly distinguishes itself from browser_press_key (keys) and type_text (native apps), so an agent can route correctly without opening schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit precondition ('browser_click the field first') and names the alternatives with the conditions that select them: browser_press_key for Enter/Tab/Escape/Backspace and type_text for native apps. Nothing is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
browser_use_tabAdopt user's tabAIdempotent
Take over a tab the user already has open, instead of opening a new one. Use this when the page is already signed in or mid-flow — a checkout, a draft, a dashboard behind SSO — and re-opening the URL would lose that state. Find the tab_id with browser_list_tabs all=true. The tab stays exactly where it is in the user's window; it is not moved into the agent's group, not activated, and not reloaded. It is never closed by cleanup — call browser_release_tab to hand it back. Ask the user before taking over a tab they are actively working in.
| Name | Required | Description | Default |
|---|---|---|---|
| tab_id | Yes | tab_id of the user's tab, from browser_list_tabs all=true | |
| session_id | No | Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=false and destructiveHint=false. The description adds substantial behavioral context: the tab stays in the user's window, is not moved, activated, or reloaded, is never closed by cleanup, and requires browser_release_tab to hand it back. This goes well beyond what annotations convey and is consistent with them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured: purpose, usage, behavior, and handoff are presented in a logical order. The description is slightly verbose but each sentence adds essential information—no redundancy. Could be trimmed slightly, but it's highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool that adopts a user's tab, the description covers the crucial context: when to use it, how to obtain the tab ID, the exact behavioral guarantees (no movement/reload), and the required release step. It also covers the safety aspect of asking the user. No missing information for an agent to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for both parameters. The description repeats the tab_id source (from browser_list_tabs) but does not add new semantic meaning beyond what the schema already states. Baseline 3 is appropriate since the schema fully documents parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action: 'take over a tab the user already has open' rather than opening a new one. Distinguishes from siblings like browser_open_tab and browser_select_tab by specifying the unique behavior of adopting an existing tab.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use: when page is signed in or mid-flow and re-opening would lose state. Provides the concrete step to find the tab_id via browser_list_tabs all=true. Also instructs to ask the user before taking over an active tab, which is a key usage constraint.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clickClickADestructive
Click an element by element_id (preferred: it uses the accessibility press action, so it works even when the element is scrolled out of view) or at absolute screen coordinates taken from a screenshot or zoom. Pass element_id or x and y, not both. Use browser_click for pages in the agent's Chrome tabs, right_click for context menus, and drag for press-move-release. The click reaches the target app for real and can trigger any action the user could, so read the target with get_app_state first. The agent's own pointer overlay moves to the target. On macOS the user's mouse pointer never moves; on Windows and Linux a target that ignores background input gets real mouse input, which moves the user's pointer, and the result then says via cursor.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Screen x coordinate in points, used together with y when no element_id is given | |
| y | No | Screen y coordinate in points, used together with x when no element_id is given | |
| element_id | No | Element id from the most recent get_app_state snapshot, e.g. e12. Preferred over coordinates. | |
| click_count | No | 1 for a single click (default), 2 for a double-click | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare a destructive, non-idempotent, non-read-only write, and the description adds context annotations cannot carry: the click reaches the app for real and may trigger any user-level action, the accessibility press fallback behavior, the agent pointer overlay, and the macOS vs Windows/Linux distinction where a target ignoring background input receives real mouse input that moves the user's pointer (surfaced via 'cursor' in the result). This is materially deeper than the hint flags.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the primary input mode and the exclusivity rule, then alternatives, then caveats — no filler sentences. However, the final sentence stacks multiple platform conditions into one long run-on clause, which costs a little readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter, all-optional mutation tool with no output schema, the description covers everything an agent needs: input-mode selection, alternatives, prerequisite state read, real-input caveat, platform pointer side effects, and the inline state-refresh option. Nothing critical is left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, but the description adds genuine semantics: an explicit mutual-exclusion rule between element_id and x/y, the reason element_id is preferred, and the id-invalidation behavior of return_state ('Its ids replace every earlier id'), which is not derivable from the schema alone. click_count and state_query semantics remain schema-only.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (click) and two concrete target mechanisms (element_id, screen coordinates), and explicitly names the siblings it is not: browser_click for Chrome tabs, right_click for context menus, drag for press-move-release. An agent can pick the right tool without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives the selection rule between the two input modes ('Pass element_id or x and y, not both'), states the preference (element_id is preferred and works when scrolled out of view), and routes to three named alternatives with the condition that selects them. It also adds a prerequisite ('read the target with get_app_state first').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clipboard_readRead clipboardARead-onlyIdempotent
Read the plain text on the system clipboard, for example after press_key cmd+c (ctrl+c off macOS) copied a selection, or when the user says they copied something for you. Prefer get_app_state, browser_snapshot or screenshot to read what is on screen; use this for text that was deliberately copied. Images and files on the clipboard are reported as no text. Read-only, but the clipboard can hold private data the user copied, such as passwords, so do not repeat it beyond what the task needs.
| Name | Required | Description | Default |
|---|---|---|---|
| max_chars | No | Maximum characters to return (default 20000, maximum 200000). Longer text is cut off with a note giving its full length. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only and idempotent behavior, so the description's safety profile is reinforced rather than duplicated. It adds useful behavioral context beyond annotations: images/files yield no text, and clipboard contents may contain private data like passwords, so output should not be repeated beyond task needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core action and then provides essential usage guidance, exclusions, edge cases, and a privacy caveat. Every sentence earns its place; there is no filler or repetition of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with one optional parameter and no output schema, the description covers the key conditions, alternatives, non-text edge cases, and privacy implications. Nothing critical is missing for an agent to decide when and how to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the sole parameter max_chars is already fully documented, including default, maximum, and truncation note. The description adds no additional parameter-specific semantics, matching the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Read the plain text on the system clipboard.' It also differentiates itself from siblings by explicitly saying to prefer get_app_state, browser_snapshot, or screenshot for on-screen content and to use this tool for text that was deliberately copied.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives concrete when-to-use examples: after press_key cmd+c / ctrl+c copies a selection, or when the user says they copied something. It also names alternatives and explicitly distinguishes on-screen reading from clipboard reading, and notes that images/files are reported as no text.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
clipboard_writeWrite clipboardADestructiveIdempotent
Replace the system clipboard with plain text, typically so a long or multi-line value can be pasted with press_key cmd+v (ctrl+v off macOS) where set_value is rejected and type_text would be slow. This tool does not paste anything itself. Side effect: whatever the user had on the clipboard is overwritten and not restored, so tell the user when you use it. Prefer set_value or type_text when they work.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Exact text to place on the clipboard, replacing its current contents |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already signal destructiveness, but the description adds crucial behavior beyond them: it overwrites the user's clipboard and does not restore it, and it tells the agent to inform the user. It also clarifies that the tool only writes the clipboard and does not perform the paste, preventing a common misuse.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, all necessary: the action, the use case, and the critical side-effect warning. The most decision-relevant facts are front-loaded, and there is no filler or repetition of schema information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter tool with no output schema, the description covers all an agent needs: what it does, when to use it, what it does not do, its side effects, and how it relates to sibling tools. Nothing essential is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single parameter is already well described. The description adds a little extra meaning by specifying plain text and mentioning the typical use case of long or multi-line values, which helps the agent understand the character of the input beyond the schema's 'Exact text.'
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Replace the system clipboard with plain text.' It also explains the intended role in a larger workflow (enabling cmd+v paste) and explicitly differentiates itself from set_value and type_text, so an agent can distinguish it from siblings without inspecting schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives concrete when-to-use context: when a long or multi-line value must be pasted via press_key and set_value is rejected or type_text would be slow. It also states a clear exclusion: 'This tool does not paste anything itself' and explicitly recommends set_value or type_text when they work.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dragDragADestructive
Press at one point, move, and release at another to drag and drop, move a slider, or select a range. Give each end as an element id or as screen coordinates; the two ends may use different forms. A drop can move or reorder items in the app, so verify the result with get_app_state. On macOS a drag must stay inside one app window and never moves the user's pointer; on Windows and Linux most drags use real mouse input, which moves the user's pointer (the result then says via cursor).
| Name | Required | Description | Default |
|---|---|---|---|
| to_x | No | Screen x to release at, used with to_y when no to_element_id is given | |
| to_y | No | Screen y to release at, used with to_x when no to_element_id is given | |
| from_x | No | Screen x to start at, used with from_y when no from_element_id is given | |
| from_y | No | Screen y to start at, used with from_x when no from_element_id is given | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. | |
| to_element_id | No | Element to release on, from get_app_state | |
| from_element_id | No | Element to start the drag on, from get_app_state |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare destructive/not-read-only/not-idempotent; the description adds genuinely new behavioral facts: macOS drags are confined to one app window and never move the user's pointer, while Windows/Linux use real mouse input that does move the pointer and reports 'via cursor'. That platform-level side-effect disclosure is exactly what annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the core action and outcomes, then the parameter-form rule, then the platform caveats. No filler, and the highest-value constraint (verify with get_app_state) is placed before the platform detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, non-idempotent interaction tool with no output schema, the description covers the action, both coordinate/element input forms, the verification path, and the platform-dependent side effects. Nothing an agent needs in order to invoke it safely is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds a rule the schema does not state: each end may be given as an element id or screen coordinates, and the two ends may use different forms. That mixed-form semantics is real added value over the per-parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a concrete verb sequence (press, move, release) and enumerates the three outcomes it covers: drag-and-drop, slider movement, and range selection. This cleanly separates it from siblings like click, scroll, and select_text, which an agent can rule out without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It tells the agent to follow up with get_app_state because a drop may move or reorder items, which is real routing guidance. It also frames the valid use cases (drop, slider, range). It stops short of explicit when-not-to-use or a named alternative for plain movement, so it is strong but not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_app_stateRead accessibility treeARead-onlyIdempotent
Read an app's accessibility tree as an indented outline in which interactive elements carry ids like [e12] that click, type_text, set_value, scroll, hover and select_text accept. Use it instead of screenshot whenever you intend to act: it is far cheaper in tokens and gives exact targets. Call it before interacting and again after the UI changes, because ids are per-snapshot. Each id is rechecked before an action: one whose element is gone or has changed fails instead of acting on something else. The outline lists what is on screen: rows of long lists and content scrolled out of view are skipped (scroll, or pass offscreen), and while a dialog, sheet or popover blocks a window only its controls are listed. When an app exposes little to accessibility, its on-screen text is read with OCR and listed with ids too. Read-only; it does not change focus.
| Name | Required | Description | Default |
|---|---|---|---|
| app | Yes | App name, bundle id, or pid exactly as reported by list_apps, or "frontmost" for the app the user is looking at | |
| ocr | No | Read on-screen text with OCR and list it with ids that click, hover, right_click and drag accept: auto (default) only when the app exposes no labelled controls to accessibility, always, or never. macOS and Windows; macOS needs Screen Recording permission. | |
| query | No | Only list elements whose role, label or value contains this text (case-insensitive). Ids stay valid. It searches the whole outline, not just the first max_elements elements, and says when the read stopped early. Use it instead of raising max_elements when you know what you are looking for. | |
| max_depth | No | Maximum nesting depth to descend (default 18). Lower it for a quick overview of a large window. | |
| offscreen | No | Also list elements scrolled out of view, such as every row of a long list (default false). Prefer scrolling or `query`: this can make the outline far larger. | |
| max_elements | No | Maximum elements to emit before the outline is truncated (default 800). With `query`, the maximum matches to list instead: a query searches past this cap. Prefer `query` over raising this. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well past the readOnly/idempotent annotations: ids are per-snapshot, stale ids fail rather than mis-clicking, offscreen rows are skipped, only blocking-dialog controls appear when a modal is up, and OCR substitutes for apps with weak accessibility exposure. It also discloses the non-effect on focus, which annotations cannot express.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Six tight sentences, each carrying distinct load: what it returns, why over screenshot, when to re-call, what ids guarantee, what is excluded, and the OCR fallback. The most decision-critical point (use this before acting) lands in sentence two.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates by describing the return format, the id scheme, truncation behavior, and the modal/offscreen omissions an agent must anticipate. Nothing needed to call it correctly on a six-parameter, read-only tool is left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is already 100%, so the baseline is 3; the description adds real guidance by recommending `query` over `max_elements` and over `offscreen`, and explaining that query searches the whole outline and preserves id validity. It does not restate enum mechanics, which is fine since the schema does.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — reading an app's accessibility tree — and immediately characterizes the output shape (indented outline with [e12] ids) and who consumes those ids. It distinguishes itself from `screenshot` and implicitly from `browser_snapshot` by framing itself as the app-level, act-before-you-click reader.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing: 'Use it instead of screenshot whenever you intend to act', 'Call it before interacting and again after the UI changes'. It also names in-parameter alternatives (scroll or pass `offscreen`, use `query` instead of raising `max_elements`) and states preconditions for the OCR path (Screen Recording permission).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
hoverHoverAIdempotent
Move the agent pointer over an element or screen position without clicking, to reveal hover menus, toolbars, tooltips or drag handles. Follow with get_app_state or screenshot to see what appeared. Use click to activate. Pass element_id or x and y, not both. On macOS the user's own mouse pointer is not moved; on Windows and Linux it may be, and the result then says via cursor.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Screen x coordinate in points, used together with y when no element_id is given | |
| y | No | Screen y coordinate in points, used together with x when no element_id is given | |
| element_id | No | Element id from the most recent get_app_state snapshot, e.g. e12 | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=true, and destructiveHint=false, so the safety profile is partly covered. The description adds genuinely non-obvious behavioral context beyond the annotations: on macOS the user's pointer is not moved, on Windows/Linux it may be, and the result reports 'via cursor'. It does not discuss any notion of limits or failure modes, so not a full 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
It is a short, front-loaded block of sentences with no filler. The primary action and its purpose come first, followed by follow-up guidance, the targeting constraint, and the platform caveat, each earning its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates by telling the agent how to observe the result ('Follow with get_app_state or screenshot to see what appeared'). Combined with the mutual-exclusion rule and platform behavior, the description is complete enough for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all five parameters, which sets a baseline of 3. The description adds a real semantic constraint not in the schema: element_id and x/y are mutually exclusive ('not both'), which clarifies how the targeting parameters interact.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb (move the agent pointer over) plus the resource (an element or screen position) and the reason (reveal hover menus, toolbars, tooltips, drag handles). It also explicitly contrasts with the sibling action by saying 'Use click to activate', so the agent can distinguish hover from click without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states when to use it (to reveal hover content), what to do afterward ('Follow with get_app_state or screenshot'), and how to escalate ('Use click to activate'). It also gives the exclusivity rule for the two targeting modes ('Pass element_id or x and y, not both'), leaving nothing to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_appsList running appsARead-onlyIdempotent
List running applications with their bundle id, pid, window count, and which one is frontmost. Call it first to learn the exact app value that get_app_state, screenshot and activate_app accept. One app can have several running instances and only some own windows, so prefer the instance that has windows. Instances that share a name each show their main window title, and a closing note lists their pids: pass the pid to choose one. Read-only: no window or input is touched.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only/idempotent, but the description adds real context beyond them: multiple instances per app, instances that own no windows, a closing note listing pids, and that no window or input is touched. This is the exact guidance needed to interpret the output correctly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, each carrying distinct information: purpose, call-order guidance, instance-selection advice, and a safety note. The controlling guidance is front-loaded with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description must characterize the return shape — and it does thoroughly (bundle id, pid, window count, frontmost flag, main window title, closing pid note). Nothing an agent needs to call or interpret it is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, so baseline is 4. The description still adds useful semantics by explaining how returned identifiers (app name, pid) should be supplied to downstream tools, exceeding the schema's empty object.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ("List running applications") and immediately enumerates the returned fields (bundle id, pid, window count, frontmost), which distinguishes it from siblings like get_app_state and list_displays.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says "Call it first to learn the exact `app` value that get_app_state, screenshot and activate_app accept" — naming three consuming siblings and the condition that selects this tool. It also advises preferring the instance that has windows and passing a pid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_displaysList displaysARead-onlyIdempotent
List every attached display with its index, resolution and position, for use with screenshot(display: N) and for interpreting screen coordinates on multi-monitor setups. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds context beyond annotations by detailing the returned data (index, resolution, position) and its intended use. The 'Read-only' remark is redundant with annotations but not contradictory.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, front-loads the core action and result, and immediately states the use cases. Every clause adds value without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a stateless, parameterless read-only tool, the description is nearly complete. It states what is returned and why. A minor gap is not explicitly noting the zero-based nature of display indices or the exact output format, but these are inferable from the list of nouns and the reference to screenshot(display: N).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters and the schema coverage is 100% (empty schema fully describes the input). With no parameters to document, the baseline of 4 applies; the description correctly focuses on output and purpose rather than parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and resource ('attached display'), and specifies what is returned: index, resolution, and position. It also explicitly ties the tool's purpose to downstream use with screenshot(display: N) and multi-monitor coordinate interpretation, making it clearly distinguishable from all sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit usage context: this is the tool to call before taking screenshots on a specific display or when working with screen coordinates on multi-monitor setups. It effectively tells an agent when to select this tool among the many siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
press_keyPress keyADestructive
Press one named key, optionally with modifiers held, e.g. key='s' modifiers=['cmd'] to save or key='return' to submit. Use it for shortcuts and navigation keys; use type_text for literal text and browser_press_key inside the agent's Chrome tabs. The key goes to the app you are working in (the one you last clicked or focused); on macOS and Windows the user can keep typing elsewhere meanwhile, so call activate_app only when nothing has targeted the app yet. Shortcuts can close windows or delete content, so confirm the target before pressing.
| Name | Required | Description | Default |
|---|---|---|---|
| key | Yes | Key name: a single character such as 's' or '/' (resolved through the current keyboard layout), or a named key: return, tab, escape, space, delete, backspace, forwarddelete, up, down, left, right, home, end, pageup, pagedown, insert, f1 to f20, punctuation names (minus, equal, leftbracket, rightbracket, backslash, semicolon, quote, comma, period, slash, grave), numpad0 to numpad9, numpadadd, numpadsubtract, numpadmultiply, numpaddivide, numpaddecimal, numpadenter | |
| modifiers | No | Modifier keys to hold while pressing: any of cmd, shift, alt, ctrl, fn. cmd maps to the Windows/Super key off macOS. | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=false, but the description adds context annotations cannot: the keystroke goes to the last-clicked/focused app, on macOS/Windows the user may be typing elsewhere concurrently, and shortcuts 'can close windows or delete content, so confirm the target before pressing.' That is genuine disclosure of risk surface and target routing, not restatement of the annotation flags.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four tightly packed sentences, front-loaded with purpose then routing then caveat, and each sentence carries distinct information. It is dense rather than padded, though the focus/multi-user sentence is long enough to slow the read slightly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter, non-idempotent, destructive input tool with a rich schema and no output schema, the description covers purpose, routing, target resolution, and the destructive caveat. The only notable gap is that it does not mention the state-returning behavior enabled by return_state beyond a brief schema-level note, which is largely handled by the parameter documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each parameter (key, modifiers, state_query, return_state) fully documented in the schema, so the baseline is 3. The description's example pairing key with modifiers adds a small amount of practical framing, but it does not explain return_state or state_query semantics beyond what the schema already says.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Press one named key, optionally with modifiers held') and immediately grounds it with two concrete examples ('cmd'+'s' to save, 'return' to submit). It also explicitly separates itself from the two closest siblings, type_text and browser_press_key, so an agent can route correctly without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit when-to-use ('shortcuts and navigation keys') and when-not with named alternatives ('use type_text for literal text and browser_press_key inside the agent's Chrome tabs'). It further adds the precondition that governs a sibling call: 'call activate_app only when nothing has targeted the app yet.'
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
right_clickRight-clickADestructive
Right-click (secondary click) an element or screen position to open its context menu. Follow with get_app_state to read the menu items, then click one. Use click for normal activation. Pass element_id or x and y, not both. The user's pointer is treated as for click.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Screen x coordinate in points, used together with y when no element_id is given | |
| y | No | Screen y coordinate in points, used together with x when no element_id is given | |
| element_id | No | Element id from the most recent get_app_state snapshot, e.g. e12 | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=false, so the safety profile is covered. The description adds the useful note that the pointer is 'treated as for click' and that ids may be replaced after return_state, but it never says the opened context menu can expose destructive actions or that nothing is committed until a menu item is clicked.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four compact sentences, front-loaded with purpose followed by usage and then parameter rules; nothing is padded. The trailing sentence about the user's pointer is the vaguest and earns the least of its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, and the description compensates by telling the agent the context menu contents must be read via get_app_state and that return_state can append fresh state in the same call, including the id-invalidation caveat. Nothing needed to invoke and interpret the call is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds the mutual-exclusion rule that the schema does not state ('Pass element_id or x and y, not both'), which genuinely changes how an agent constructs the call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (right-click / secondary click) plus the resource (an element or screen position) and the effect (opens its context menu). It also distinguishes itself from the sibling 'click' by naming it and assigning it the normal-activation case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives the when-to-use alternative ('Use click for normal activation') and the follow-on workflow ('Follow with get_app_state to read the menu items, then click one'). An agent knows exactly which tool to pick and what to do next.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotScreenshotARead-onlyIdempotent
Capture an app's largest window, or a whole display, as an image. The result text states the capture's screen origin and pixels-per-point so an image pixel can be converted into click or hover coordinates. Prefer get_app_state for interaction, which is cheaper and returns clickable element ids; use screenshot to verify an outcome or to see content the accessibility tree cannot describe (canvas, video, custom drawing), and zoom to read small text. Read-only; the captured window is not raised or focused.
| Name | Required | Description | Default |
|---|---|---|---|
| app | No | App name, bundle id, or pid exactly as reported by list_apps. Captures that app's largest window. Provide either app or display, or "frontmost" for the app the user is looking at | |
| format | No | Image encoding (default png). Use jpeg for live remote viewing. | |
| display | No | 0-based display index from list_displays. Captures the whole display instead of an app window. | |
| quality | No | JPEG quality from 1 to 100 (default 55). Ignored for png. Raise it when small text must stay sharp. | |
| max_width | No | Downscale the image to this width in pixels (default 1400). Lower it to save tokens. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent and non-destructive, but the description adds traits they do not cover: the captured window is not raised or focused, and the result text carries screen origin plus pixels-per-point so image pixels convert to click/hover coordinates. That is genuine extra behavioral context beyond the structured hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three densely-packed sentences with the primary action, the alternative tool, and the side-effect disclosure front-loaded in that order. No filler and every clause carries actionable information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly compensates by explaining the result text's coordinate-conversion data. Combined with the full schema for parameters and annotations for the safety profile, an agent has everything needed to call and interpret this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents app/display/format/quality/max_width including defaults and valid values; that establishes the baseline of 3. The description adds only the loose hint that zoom helps read small text (related to quality/max_width) and otherwise does not extend parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: capture an app's largest window or a whole display as an image. It also names the output semantics (screen origin, pixels-per-point) and explicitly distinguishes itself from sibling get_app_state and zoom, so an agent can pick it without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to prefer get_app_state for interaction because it is cheaper and returns clickable element ids, and defines exactly when screenshot is the right choice: verifying an outcome or seeing content the accessibility tree cannot describe, using zoom for small text. This is a full when/when-not/alternatives routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scrollScrollA
Scroll up, down, left or right by a number of lines, over element_id when given so the right pane scrolls. Use it to bring off-screen content into view before get_app_state or screenshot. It only scrolls; nothing is clicked or selected. On macOS it scrolls the target app in the background without moving the user's pointer. On Windows and Linux, without element_id it scrolls whatever is under the user's pointer, and an element with no background route gets the real pointer moved onto it (the result then says via cursor).
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | Screen x to scroll over. Remote control only: the pointer moves there first. Ignored otherwise. | |
| y | No | Screen y to scroll over. Remote control only: the pointer moves there first. Ignored otherwise. | |
| amount | No | Number of scroll lines, 1 to 100 (default 5) | |
| direction | No | Scroll direction (default down) | |
| element_id | No | Element to scroll, from get_app_state; the nearest scrollable area around it moves. Omit to scroll the last inspected app (macOS) or whatever is under the pointer (Windows, Linux). | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by disclosing platform-specific side effects: macOS scrolls the target app in the background without moving the pointer, while Windows/Linux scroll under the pointer and may move the real cursor onto an element with no background route (with a 'via cursor' marker in the result). It also discloses that nothing is clicked or selected, which the readOnlyHint=false / destructiveHint=false annotations alone do not convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and direction, then layered with usage and platform caveats in a compact sequence. It is dense but every clause carries real information; the platform-specific sentences are long but necessary for a cross-platform tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and 7 optional parameters, the description supplies the behavioral context an agent needs: what triggers scrolling, what happens on each platform, and what the 'via cursor' result marker means. Combined with the fully-covered schema, nothing essential for correct invocation is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so x, y, amount, direction, element_id, state_query, and return_state are already fully documented. The description adds only marginal meaning, mostly restating element_id's conditional behavior ('over element_id when given') that the schema already spells out for the omit case.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (scroll) plus direction and line count, and scopes the resource to element_id or the app under the pointer. It is immediately distinguishable from siblings like click, drag, and get_app_state, and it explicitly says 'It only scrolls; nothing is clicked or selected.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a concrete when-to-use ('bring off-screen content into view before get_app_state or screenshot') and names the follow-up tools that consume the result. The boundary statement ('only scrolls; nothing is clicked or selected') tells the agent when a different sibling (click/select) is required.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
select_textSelect textAIdempotent
Select a character range inside a text element through the accessibility API, for example to copy part of a value or to replace just that part with type_text. Defaults to selecting from start to the end of the value. Use set_value to replace the whole value instead. Only the selection changes; the text is not modified.
| Name | Required | Description | Default |
|---|---|---|---|
| start | No | Zero-based character offset to start the selection at (default 0) | |
| length | No | Number of characters to select (default: through the end of the value) | |
| element_id | Yes | Text element to select in, from get_app_state | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true, destructiveHint=false and readOnlyHint=false. The description earns credit by resolving the tension that readOnlyHint=false creates: it clarifies that only the selection changes and the underlying text is not modified, which is the key behavioral fact an agent needs. It does not discuss permissions or error cases, so it stops short of 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the action and scope, then the sibling alternative, then the side-effect disclaimer. No filler and nothing redundant between sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 5-parameter tool with full schema coverage and no output schema, the description covers purpose, default range, side effect, and sibling routing. It does not explain what the return_state snapshot will look like, but the schema already describes that, leaving only minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so every parameter (including start/length defaults and the return_state mechanics) is already documented in the schema. The description only restates the start-to-end default, adding no syntax or semantics beyond that. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (select) and resource (a character range inside a text element), plus the mechanism (accessibility API). It explicitly separates itself from the adjacent siblings type_text and set_value, so an agent can route without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives both a positive use case (copy part of a value, or set up a partial replacement via type_text) and the explicit alternative plus its selection condition ('Use set_value to replace the whole value instead'). Default behavior (start -> end) is stated so the agent knows how to call it minimally.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
set_valueSet field valueADestructiveIdempotent
Replace a text field's entire contents in one step through the accessibility API, without keystrokes. Prefer it over type_text for long values or when the field already holds text; fall back to click plus type_text if the field rejects it, which the result reports. The previous value is discarded.
| Name | Required | Description | Default |
|---|---|---|---|
| value | Yes | New complete value for the field | |
| element_id | Yes | Text field to set, from get_app_state | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true, so the bar is lower. The description reinforces this by stating the previous value is discarded and that failures surface in the result, adding useful context beyond the annotation flags.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences with zero waste: purpose, routing rule, and the destructive caveat. Front-loaded and well ordered.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-param mutation tool with full schema coverage and no output schema, the description covers purpose, alternatives, fallback behavior, and the destructive consequence. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all four parameters are already documented in the schema. The description adds no parameter-level detail beyond what the schema provides; baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Replace a text field's entire contents') plus the mechanism ('through the accessibility API, without keystrokes'), which cleanly distinguishes it from the sibling type_text.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit routing: prefer over type_text for long values or pre-filled fields; fall back to click plus type_text if rejected, and it names where the rejection is reported ('which the result reports').
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
type_textType textADestructive
Type literal text as keystrokes into the field that currently has focus, optionally focusing element_id first. Use it for short entries and for fields that reject set_value; use set_value to replace a long value in one step, and press_key for shortcuts or keys such as return and tab. Text is inserted at the caret without clearing what is already there. It does not take over the user's keyboard: on macOS and Windows the text goes to the app you are working in, not the window the user is typing in, so do not call activate_app first. Typing into password fields is refused by default (see COMPUTER_USE_ALLOW_SECURE_FIELD_INPUT).
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | Exact text to type, character by character | |
| element_id | No | Element to focus before typing, from get_app_state. Omit to type into whatever currently has focus. | |
| state_query | No | With return_state, list only elements whose role, label or value contains this text, like get_app_state's query | |
| return_state | No | After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=true, readOnlyHint=false and idempotentHint=false; the description goes well beyond them by disclosing caret-insertion semantics (no clearing), keyboard non-hijack behavior on macOS/Windows, the activate_app exclusion, and the password-field refusal with its env-var escape hatch. This is exactly the extra behavioral context annotations cannot carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action, then alternatives, then behavioral caveats — a logical order with no filler sentences. It runs somewhat long across five sentences, but each earns its place by adding distinct constraints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description explains what return_state returns and that ids are replaced, covering the result-inspection path. Combined with safety-relevant annotations and the accessibility/password caveats, an agent has everything needed to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3, but the description reinforces element_id behavior ('optionally focusing element_id first') and return_state/state_query semantics ('a fresh get_app_state... its ids replace every earlier id'). It adds some operational meaning beyond the schema text, though most param detail lives in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('type literal text as keystrokes into the field that currently has focus') and immediately differentiates from siblings set_value and press_key by naming what each is for. An agent can tell exactly what this does versus click/select_text/set_value without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit when-to-use ('short entries', 'fields that reject set_value') and named alternatives with their conditions ('use set_value to replace a long value in one step, and press_key for shortcuts or keys such as return and tab'). Also gives a when-not hint: 'do not call activate_app first'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
waitWaitARead-onlyIdempotent
Pause before the next action so the UI can catch up: page loads, animations, dialogs opening, apps launching. Follow with get_app_state or screenshot to confirm the new state instead of guessing. Sends no input.
| Name | Required | Description | Default |
|---|---|---|---|
| seconds | No | Seconds to wait (default 1, maximum 30) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only, idempotent, and non-destructive behavior. The description adds useful behavioral context beyond that: the wait exists to let UI transitions settle, and it sends no input. This enriches the annotation profile without contradicting it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two purposeful sentences front-load the core purpose and usage context, then add the verification follow-up and the behavioral constraint. Every sentence earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple no-input pausing tool with strong annotations and a fully described schema, the description covers why, when, and how to confirm the result. Nothing an agent needs to correctly invoke this tool is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the only parameter (seconds) is already fully documented with default and maximum. The description does not need to repeat this, but it also adds no additional nuance about how the parameter interacts with wait behavior. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Pause') and gives concrete use cases (page loads, animations, dialogs opening, apps launching), so an agent can tell exactly what the tool does. It also separates this action-oriented tool from siblings like get_app_state or screenshot by noting it 'sends no input.'
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It clearly states when to use the tool: before the next action when the UI needs to catch up. It also advises following up with get_app_state or screenshot to confirm the new state instead of guessing, giving actionable post-wait guidance. It does not explicitly discuss when not to use it, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
zoomZoom into regionARead-onlyIdempotent
Capture one region of the screen at full resolution, to read small text, dense tables, file names or tiny controls that a normal screenshot blurs. Give the region as two corners in screen coordinates (the same space click uses); the result text explains how to map pixels in the zoomed image back to screen coordinates. Use screenshot for a whole window and get_app_state when the text is exposed by accessibility. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| x0 | Yes | Left edge, screen coordinates | |
| x1 | Yes | Right edge, screen coordinates | |
| y0 | Yes | Top edge, screen coordinates | |
| y1 | Yes | Bottom edge, screen coordinates | |
| max_width | No | Downscale the zoomed image to this width in pixels (default 1400) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive behavior, so the description does not need to repeat that. It adds useful behavioral context: the coordinate space is the same as click, and the result text explains how to map pixels back to screen coordinates. This goes beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences: purpose/use cases, input format plus result behavior, and alternative routing. There is no filler or repetition; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only capture tool with fully documented parameters, the description covers why to use it, when to avoid it, how to specify input, and what the result will indicate. No output schema exists, but the description does disclose the essential result behavior, so the agent has what it needs to call the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3; the description adds meaningful extra context by clarifying that the region is defined by two corners in the same coordinate space that click uses. It also explains why that matters by mentioning pixel-to-screen mapping in the result.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Capture one region of the screen at full resolution.' It also names concrete use cases (small text, dense tables, file names, tiny controls) and is clearly differentiated from the sibling screenshot tool by emphasizing regions at full resolution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly tells the agent when to use this tool versus alternatives: 'Use screenshot for a whole window and get_app_state when the text is exposed by accessibility.' This gives direct routing guidance beyond mere inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
2 tool updates
v0.7.3- Changed
activate_app1 field changed- added
Input schema / properties / window_idAdded value: +{ + "description": "Windows only: bring this one window forward instead of the app's frontmost window, for an app with several windows such as Chrome. Use the window_id from a get_app_state window line; it must belong to app. The result names the window in front afterwards.", + "type": "integer" +}
- Changed
get_app_state2 fields changed- changed
Input schema / properties / max_elements / descriptionPrevious value: -"Maximum elements to emit before the outline is truncated (default 800). Prefer `query` over raising this."New value: +"Maximum elements to emit before the outline is truncated (default 800). With `query`, the maximum matches to list instead: a query searches past this cap. Prefer `query` over raising this." - changed
Input schema / properties / query / descriptionPrevious value: -"Only list elements whose role, label or value contains this text (case-insensitive). Ids stay valid. Use it instead of raising max_elements when you know what you are looking for."New value: +"Only list elements whose role, label or value contains this text (case-insensitive). Ids stay valid. It searches the whole outline, not just the first max_elements elements, and says when the read stopped early. Use it instead of raising max_elements when you know what you are looking for."
15 tool updates
v0.7.1- Changed
browser_click3 fields changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones."New value: +"Append a current snapshot and readiness status after acting. Without wait_for_selector, only a short pause and bounded navigation wait are used; SPA readiness is not guaranteed. Snapshot indices replace earlier ones. A timeout does not mean the action failed: re-observe with browser_snapshot before repeating it." - added
Input schema / properties / wait_for_selectorAdded value: +{ + "description": "Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI.", + "maxLength": 2000, + "minLength": 1, + "type": "string" +} - added
Input schema / properties / wait_timeout_msAdded value: +{ + "description": "Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action.", + "maximum": 10000, + "minimum": 0, + "type": "integer" +}
- Changed
browser_navigate3 fields changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones."New value: +"Append a current snapshot and readiness status after acting. Without wait_for_selector, only a short pause and bounded navigation wait are used; SPA readiness is not guaranteed. Snapshot indices replace earlier ones. A timeout does not mean the action failed: re-observe with browser_snapshot before repeating it." - added
Input schema / properties / wait_for_selectorAdded value: +{ + "description": "Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI.", + "maxLength": 2000, + "minLength": 1, + "type": "string" +} - added
Input schema / properties / wait_timeout_msAdded value: +{ + "description": "Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action.", + "maximum": 10000, + "minimum": 0, + "type": "integer" +}
- Changed
browser_press_key3 fields changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones."New value: +"Append a current snapshot and readiness status after acting. Without wait_for_selector, only a short pause and bounded navigation wait are used; SPA readiness is not guaranteed. Snapshot indices replace earlier ones. A timeout does not mean the action failed: re-observe with browser_snapshot before repeating it." - added
Input schema / properties / wait_for_selectorAdded value: +{ + "description": "Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI.", + "maxLength": 2000, + "minLength": 1, + "type": "string" +} - added
Input schema / properties / wait_timeout_msAdded value: +{ + "description": "Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action.", + "maximum": 10000, + "minimum": 0, + "type": "integer" +}
- Changed
browser_snapshot4 fields changed- added
Input schema / properties / limitAdded value: +{ + "description": "Maximum controls returned per page (default 250, maximum 250).", + "maximum": 250, + "minimum": 1, + "type": "integer" +} - added
Input schema / properties / offsetAdded value: +{ + "description": "Control offset for pagination (default 0). Use the continuation offset from the previous result on an unchanged UI.", + "maximum": 2147483647, + "minimum": 0, + "type": "integer" +} - added
Input schema / properties / wait_for_selectorAdded value: +{ + "description": "Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI.", + "maxLength": 2000, + "minLength": 1, + "type": "string" +} - added
Input schema / properties / wait_timeout_msAdded value: +{ + "description": "Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action.", + "maximum": 10000, + "minimum": 0, + "type": "integer" +}
- Changed
browser_type3 fields changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones."New value: +"Append a current snapshot and readiness status after acting. Without wait_for_selector, only a short pause and bounded navigation wait are used; SPA readiness is not guaranteed. Snapshot indices replace earlier ones. A timeout does not mean the action failed: re-observe with browser_snapshot before repeating it." - added
Input schema / properties / wait_for_selectorAdded value: +{ + "description": "Wait for any visible element matching this CSS selector in the top-level document (not frames or shadow roots). On actions requires return_state=true. Returns readiness met, timeout or error; only this condition is checked, not general app readiness. Prefer a selector specific to the expected new UI.", + "maxLength": 2000, + "minLength": 1, + "type": "string" +} - added
Input schema / properties / wait_timeout_msAdded value: +{ + "description": "Bound for wait_for_selector in milliseconds (default 8000, maximum 10000; 0 checks once). Requires wait_for_selector. Timeout preserves action success and returns current state. Use browser_snapshot with the same wait to re-observe without repeating an action.", + "maximum": 10000, + "minimum": 0, + "type": "integer" +}
- Changed
click1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
drag1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
get_app_state2 fields changed- added
Input schema / properties / ocrAdded value: +{ + "description": "Read on-screen text with OCR and list it with ids that click, hover, right_click and drag accept: auto (default) only when the app exposes no labelled controls to accessibility, always, or never. macOS and Windows; macOS needs Screen Recording permission.", + "enum": [ + "auto", + "always", + "never" + ], + "type": "string" +} - added
Input schema / properties / offscreenAdded value: +{ + "description": "Also list elements scrolled out of view, such as every row of a long list (default false). Prefer scrolling or `query`: this can make the outline far larger.", + "type": "boolean" +}
- Changed
hover1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
press_key1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
right_click1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
scroll1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
select_text1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
set_value1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
- Changed
type_text1 field changed- changed
Input schema / properties / return_state / descriptionPrevious value: -"After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."New value: +"After acting, wait for the app to finish reacting (up to 2 s) and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id."
17 tool updates
v0.5.2- Changed
browser_click1 field changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones.", + "type": "boolean" +}
- Changed
browser_navigate1 field changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones.", + "type": "boolean" +}
- Changed
browser_press_key1 field changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones.", + "type": "boolean" +}
- Added
browser_read - Added
browser_request_credentials - Changed
browser_type1 field changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait for the page to settle (and finish loading, if the action navigated) and append a fresh browser_snapshot of this tab, so you can pick the next index in the same call. Its indices replace earlier ones.", + "type": "boolean" +}
- Changed
click2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
drag2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
get_app_state1 field changed- changed
Input schema / properties / app / descriptionPrevious value: -"App name, bundle id, or pid exactly as reported by list_apps"New value: +"App name, bundle id, or pid exactly as reported by list_apps, or \"frontmost\" for the app the user is looking at"
- Changed
hover2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
press_key2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
right_click2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
screenshot2 fields changed- changed
Input schema / properties / app / descriptionPrevious value: -"App name, bundle id, or pid exactly as reported by list_apps. Captures that app's largest window. Provide either app or display."New value: +"App name, bundle id, or pid exactly as reported by list_apps. Captures that app's largest window. Provide either app or display, or \"frontmost\" for the app the user is looking at" - added
Input schema / properties / qualityAdded value: +{ + "description": "JPEG quality from 1 to 100 (default 55). Ignored for png. Raise it when small text must stay sharp.", + "maximum": 100, + "minimum": 1, + "type": "integer" +}
- Changed
scroll2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
select_text2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
set_value2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
- Changed
type_text2 fields changed- added
Input schema / properties / return_stateAdded value: +{ + "description": "After acting, wait briefly for the UI to settle and append a fresh get_app_state of the app you last read, so you can check the result and pick the next target in the same call. Its ids replace every earlier id.", + "type": "boolean" +} - added
Input schema / properties / state_queryAdded value: +{ + "description": "With return_state, list only elements whose role, label or value contains this text, like get_app_state's query", + "type": "string" +}
12 tool updates
v0.4.3- Changed
browser_click1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_close_all_tabs1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_close_tab1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_list_tabs1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_navigate1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_open_tab1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_press_key1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_release_tab1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_select_tab1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_snapshot1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_type1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
- Changed
browser_use_tab1 field changed- added
Input schema / properties / session_idAdded value: +{ + "description": "Stable task or thread ID. Pass the same value on every browser call for this task. Different IDs isolate tabs and cleanup within one MCP process. Omit for the default process session.", + "maxLength": 128, + "minLength": 1, + "type": "string" +}
1 tool update
v0.4.2- Changed
scroll2 fields changed- added
Input schema / properties / xAdded value: +{ + "description": "Screen x to scroll over. Remote control only: the pointer moves there first. Ignored otherwise.", + "type": "number" +} - added
Input schema / properties / yAdded value: +{ + "description": "Screen y to scroll over. Remote control only: the pointer moves there first. Ignored otherwise.", + "type": "number" +}
4 tool updates
v0.4.0- Added
clipboard_read - Added
clipboard_write - Changed
press_key1 field changed- changed
Input schema / properties / key / descriptionPrevious value: -"Key name: a single character such as 's', or a named key such as return, tab, escape, space, delete, backspace, up, down, left, right, home, end, pageup, pagedown"New value: +"Key name: a single character such as 's' or '/' (resolved through the current keyboard layout), or a named key: return, tab, escape, space, delete, backspace, forwarddelete, up, down, left, right, home, end, pageup, pagedown, insert, f1 to f20, punctuation names (minus, equal, leftbracket, rightbracket, backslash, semicolon, quote, comma, period, slash, grave), numpad0 to numpad9, numpadadd, numpadsubtract, numpadmultiply, numpaddivide, numpaddecimal, numpadenter"
- Changed
scroll2 fields changed- changed
Input schema / properties / amount / descriptionPrevious value: -"Number of scroll lines (default 5)"New value: +"Number of scroll lines, 1 to 100 (default 5)" - changed
Input schema / properties / element_id / descriptionPrevious value: -"Element to position the pointer over before scrolling, from get_app_state. Omit to scroll at the current pointer position."New value: +"Element to scroll, from get_app_state; the nearest scrollable area around it moves. Omit to scroll the last inspected app (macOS) or whatever is under the pointer (Windows, Linux)."
23 tool updates
v0.3.1- Changed
activate_app1 field changed- added
Input schema / properties / app / descriptionAdded value: +"App name, bundle id, or pid exactly as reported by list_apps"
- Changed
browser_click4 fields changed- changed
Input schema / properties / index / descriptionPrevious value: -"Element index from browser_snapshot"New value: +"Element index from the latest browser_snapshot of this tab. Preferred over coordinates." - added
Input schema / properties / tab_id / descriptionAdded value: +"tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs" - added
Input schema / properties / x / descriptionAdded value: +"Page x coordinate in CSS pixels, used together with y when no index is given" - added
Input schema / properties / y / descriptionAdded value: +"Page y coordinate in CSS pixels, used together with x when no index is given"
- Changed
browser_close_tab2 fields changed- changed
Input schema / properties / index / descriptionPrevious value: -"1-based index, fallback mode only"New value: +"1-based position within the agent's tabs; fallback mode only, when tab_id is unavailable" - changed
Input schema / properties / tab_id / descriptionPrevious value: -"From browser_list_tabs"New value: +"tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs"
- Changed
browser_navigate2 fields changed- added
Input schema / properties / tab_id / descriptionAdded value: +"tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs" - added
Input schema / properties / url / descriptionAdded value: +"Absolute URL to load in the tab"
- Changed
browser_open_tab1 field changed- changed
Input schema / properties / url / descriptionPrevious value: -"URL to open (default about:blank)"New value: +"Absolute URL to open (default about:blank)"
- Changed
browser_press_key2 fields changed- added
Input schema / properties / key / descriptionAdded value: +"Key to press" - added
Input schema / properties / tab_id / descriptionAdded value: +"tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs"
- Changed
browser_release_tab1 field changed- added
Input schema / properties / tab_id / descriptionAdded value: +"tab_id of a tab previously adopted with browser_use_tab"
- Changed
browser_select_tab2 fields changed- changed
Input schema / properties / index / descriptionPrevious value: -"1-based index, fallback mode only"New value: +"1-based position within the agent's tabs; fallback mode only, when tab_id is unavailable" - changed
Input schema / properties / tab_id / descriptionPrevious value: -"From browser_list_tabs"New value: +"tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs"
- Changed
browser_snapshot1 field changed- changed
Input schema / properties / tab_id / descriptionPrevious value: -"From browser_open_tab or browser_list_tabs"New value: +"tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs"
- Changed
browser_type2 fields changed- added
Input schema / properties / tab_id / descriptionAdded value: +"tab_id of one of the agent's tabs, from browser_open_tab or browser_list_tabs" - added
Input schema / properties / text / descriptionAdded value: +"Exact text to type into the focused field"
- Changed
browser_use_tab1 field changed- changed
Input schema / properties / tab_id / descriptionPrevious value: -"From browser_list_tabs all=true"New value: +"tab_id of the user's tab, from browser_list_tabs all=true"
- Changed
click4 fields changed- changed
Input schema / properties / click_count / descriptionPrevious value: -"1 for single, 2 for double-click"New value: +"1 for a single click (default), 2 for a double-click" - changed
Input schema / properties / element_id / descriptionPrevious value: -"Element id from get_app_state, e.g. e12"New value: +"Element id from the most recent get_app_state snapshot, e.g. e12. Preferred over coordinates." - added
Input schema / properties / x / descriptionAdded value: +"Screen x coordinate in points, used together with y when no element_id is given" - added
Input schema / properties / y / descriptionAdded value: +"Screen y coordinate in points, used together with x when no element_id is given"
- Changed
drag6 fields changed- added
Input schema / properties / from_element_id / descriptionAdded value: +"Element to start the drag on, from get_app_state" - added
Input schema / properties / from_x / descriptionAdded value: +"Screen x to start at, used with from_y when no from_element_id is given" - added
Input schema / properties / from_y / descriptionAdded value: +"Screen y to start at, used with from_x when no from_element_id is given" - added
Input schema / properties / to_element_id / descriptionAdded value: +"Element to release on, from get_app_state" - added
Input schema / properties / to_x / descriptionAdded value: +"Screen x to release at, used with to_y when no to_element_id is given" - added
Input schema / properties / to_y / descriptionAdded value: +"Screen y to release at, used with to_x when no to_element_id is given"
- Changed
get_app_state3 fields changed- changed
Input schema / properties / app / descriptionPrevious value: -"App name, bundle id, or pid"New value: +"App name, bundle id, or pid exactly as reported by list_apps" - changed
Input schema / properties / max_depth / descriptionPrevious value: -"Max tree depth (default 18)"New value: +"Maximum nesting depth to descend (default 18). Lower it for a quick overview of a large window." - changed
Input schema / properties / max_elements / descriptionPrevious value: -"Max elements to emit (default 800)"New value: +"Maximum elements to emit before the outline is truncated (default 800). Prefer `query` over raising this."
- Changed
hover3 fields changed- changed
Input schema / properties / element_id / descriptionPrevious value: -"Element id from get_app_state"New value: +"Element id from the most recent get_app_state snapshot, e.g. e12" - added
Input schema / properties / x / descriptionAdded value: +"Screen x coordinate in points, used together with y when no element_id is given" - added
Input schema / properties / y / descriptionAdded value: +"Screen y coordinate in points, used together with x when no element_id is given"
- Changed
press_key2 fields changed- added
Input schema / properties / key / descriptionAdded value: +"Key name: a single character such as 's', or a named key such as return, tab, escape, space, delete, backspace, up, down, left, right, home, end, pageup, pagedown" - changed
Input schema / properties / modifiers / descriptionPrevious value: -"Any of cmd, shift, alt, ctrl, fn. cmd maps to the Windows/Super key off macOS."New value: +"Modifier keys to hold while pressing: any of cmd, shift, alt, ctrl, fn. cmd maps to the Windows/Super key off macOS."
- Changed
right_click3 fields changed- added
Input schema / properties / element_id / descriptionAdded value: +"Element id from the most recent get_app_state snapshot, e.g. e12" - added
Input schema / properties / x / descriptionAdded value: +"Screen x coordinate in points, used together with y when no element_id is given" - added
Input schema / properties / y / descriptionAdded value: +"Screen y coordinate in points, used together with x when no element_id is given"
- Changed
screenshot3 fields changed- changed
Input schema / properties / app / descriptionPrevious value: -"App name, bundle id, or pid"New value: +"App name, bundle id, or pid exactly as reported by list_apps. Captures that app's largest window. Provide either app or display." - changed
Input schema / properties / display / descriptionPrevious value: -"Capture a whole display by index (see list_displays) instead of an app window"New value: +"0-based display index from list_displays. Captures the whole display instead of an app window." - changed
Input schema / properties / max_width / descriptionPrevious value: -"Downscale to this width in pixels (default 1400)"New value: +"Downscale the image to this width in pixels (default 1400). Lower it to save tokens."
- Changed
scroll3 fields changed- changed
Input schema / properties / amount / descriptionPrevious value: -"Scroll lines (default 5)"New value: +"Number of scroll lines (default 5)" - added
Input schema / properties / direction / descriptionAdded value: +"Scroll direction (default down)" - added
Input schema / properties / element_id / descriptionAdded value: +"Element to position the pointer over before scrolling, from get_app_state. Omit to scroll at the current pointer position."
- Changed
select_text3 fields changed- added
Input schema / properties / element_id / descriptionAdded value: +"Text element to select in, from get_app_state" - changed
Input schema / properties / length / descriptionPrevious value: -"Characters to select (default: to end)"New value: +"Number of characters to select (default: through the end of the value)" - changed
Input schema / properties / start / descriptionPrevious value: -"Start offset (default 0)"New value: +"Zero-based character offset to start the selection at (default 0)"
- Changed
set_value2 fields changed- added
Input schema / properties / element_id / descriptionAdded value: +"Text field to set, from get_app_state" - added
Input schema / properties / value / descriptionAdded value: +"New complete value for the field"
- Changed
type_text2 fields changed- changed
Input schema / properties / element_id / descriptionPrevious value: -"Focus this element before typing"New value: +"Element to focus before typing, from get_app_state. Omit to type into whatever currently has focus." - added
Input schema / properties / text / descriptionAdded value: +"Exact text to type, character by character"
- Changed
wait1 field changed- changed
Input schema / properties / seconds / descriptionPrevious value: -"Seconds to wait (default 1, max 30)"New value: +"Seconds to wait (default 1, maximum 30)"
28 tool updates
v0.3.0- First observed
activate_app - First observed
browser_click - First observed
browser_close_all_tabs - First observed
browser_close_tab - First observed
browser_list_tabs - First observed
browser_navigate - First observed
browser_open_tab - First observed
browser_press_key - First observed
browser_release_tab - First observed
browser_select_tab - First observed
browser_snapshot - First observed
browser_type - First observed
browser_use_tab - First observed
click - First observed
drag - First observed
get_app_state - First observed
hover - First observed
list_apps - First observed
list_displays - First observed
press_key - First observed
right_click - First observed
screenshot - First observed
scroll - First observed
select_text - First observed
set_value - First observed
type_text - First observed
wait - First observed
zoom
TDQS
Scored across 32 tools
The browser-prefixed tools are cleanly separated from native-app tools, and descriptions explicitly explain when to use browser_click versus click, browser_type versus type_text, and browser_press_key versus press_key. However, there is still conceptual overlap around text entry, clicking, and reading state across browser and native contexts that an agent must track carefully, and tools like set_value, type_text, browser_type, and clipboard_write all touch text input.
The browser tools follow a consistent browser_verb pattern, but the native-app tools mix bare verbs (click, drag, hover), noun-like names (screenshot, zoom), and get/set/list forms (get_app_state, set_value, list_apps, list_displays). The split convention is meaningful, but within the native-app set naming is only somewhat predictable.
With 32 tools, the surface is heavy for a single server, though the domain realistically spans browser tab management, page reading/interaction, native app accessibility, screenshots, and clipboard access. Some redundancy exists between browser-specific and native-app analogues, suggesting consolidation could reduce the count, but most tools do correspond to distinct operations.
Coverage is broad: browser tabs, navigation, snapshots, clicks, typing, credentials, native app state, screenshots, zoom, drag, clipboard, and cleanup are all represented. Minor gaps exist around richer browser interaction such as file uploads or full window management, but the practical automation surface is largely complete.
Maintenance
Related MCP Connectors
MCP server connecting AI agents to 100+ apps (Gmail, Slack, Notion, GitHub) via one-click OAuth.
Use your Mac, Windows or Linux computer from ChatGPT, Claude or Codex: files, commands, documents.
Hosted Web Crawler MCP server for Claude, ChatGPT and Cursor, with each action switched per account.
An agent-first office suite Claude & ChatGPT read and write over one MCP URL.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceAn open-source MCP server for macOS and Windows that provides native desktop control via Accessibility APIs, OCR, and Chrome CDP. It enables AI agents to interact with applications, manage browser sessions, and automate workflows with high-speed native UI actions.28 npm15AGPL 3.0
- AlicenseNot gradedqualityBmaintenanceAn MCP server that gives any AI assistant eyes and hands on your desktop — screenshots, clicking, typing, OCR, window management, accessibility-tree queries, workflow recording.5Apache 2.0
- AlicenseNot gradedqualityAmaintenanceAn MCP server that gives AI agents real OS-level control of macOS, enabling them to click real buttons, type real keys, and observe rendered screens just like a human would.44 PyPI1MIT
- FlicenseNot gradedqualityDmaintenancemacOS MCP server that enables AI agents to directly control the host OS, including mouse, keyboard, windows, files, and accessibility automation for computer-use workflows.1-