computer-use
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| PATH | No | PATH environment variable used to locate executables (for example, /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin). |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
Unified Computer Use & Multi-Window/Multi-Browser MCP Server (Codex + Antigravity Extension Parity).
Core Capabilities & Best Practices:
Non-Intrusive Background Control:
Prefer element_index actions (from get_window_state / browser_tab_state) over raw (x, y) coordinates.
Element-indexed AX actions on macOS windows and CDP actions on browser tabs operate in the background without stealing the user's mouse cursor or focus.
Native paste automatically snapshots and restores the user's macOS clipboard (supports "text", "md", and "html"); browser paste dispatches ClipboardEvent directly in the tab without touching the OS clipboard.
Extension-First Browser Control & Tab Groups:
Live Chromium browsers (Brave, Chrome, Edge, Arc, Vivaldi, Opera) with the "Antigravity in Chrome" unpacked extension are controlled natively via Extension CDP without requiring --remote-debugging-port.
Agent-created tabs open in a dedicated color-coded Chrome Tab Group (default "Antigravity" in cyan; customize via group_title / group_color in browser_open_tab or browser_manage).
User tabs inspected or acted on are claimed in-place (never pulled into the agent group and never closed on finalize_session). Call browser_manage action="finalize_session" when done to close scratch tabs and detach debuggers cleanly.
If CAPTCHA / anti-bot protection appears ([HANDOFF REQUIRED: ...]), NEVER attempt to solve it — call browser_manage action="mark_tab" status="handoff" and ask the user.
Multi-Window & Multi-Tab One-Click Control:
Use manage_windows to tile, grid, quadrant-snap, cascade, move, resize, minimize, raise, or close multiple macOS windows in a single call.
Use browser_open_tab with urls: [...] to open multiple grouped background tabs simultaneously.
Use batch_execute or js (cua_repl) to control multiple macOS app windows AND multiple browser tabs in parallel in one single tool call.
Smart Accessibility Diffing & Rich DevTools Inspection:
Repeated get_window_state / browser_tab_state calls return compact line diffs so context stays clean. Pass disable_diff: true whenever you want the full tree.
browser_tab_action supports Two-Tier upload_files (including DataTransfer fallback & file-chooser interception), wait_for_download / list_downloads, non-blocking JS dialogs (get_dialog / handle_dialog), console logs, network requests/bodies, cookies, and localStorage/sessionStorage.
Daintier Pill-Free 1:1 Virtual Cursor & Unified Coordinates:
Every macOS window and browser tab displays a sleek 0.72x-scale click-through Codex AgentCursor (240Hz Velocity-Verlet Scoot & curved Bezier glide, FogCursorStyle sky-blue mist, daintier click ripples, drag trail, and element highlight frame — zero badge pill clutter).
AX element @(x,y WxH) bounds, isolated window screenshots, and window_action (x, y) share a unified 1:1 window-relative point space.
Screenshots omit the cursor and active-tab Take Over indicator by default; pass include_cursor: true to composite the live cursor.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| get_stateA | Inspect the unified desktop state in one call: macOS permissions, displays, running/installed apps, all open macOS windows with bounds & IDs, all browsers with their open tabs, and active virtual cursors. |
| list_appsB | List macOS applications (running apps and/or installed /Applications bundles). |
| launch_appA | Launch one or multiple macOS applications by name, bundle ID, or .app path (launches in the background by default without stealing focus unless activate=true). |
| list_windowsA | List open macOS windows with their window IDs, owning apps, titles, bounds, and minimized/on-screen status. |
| manage_windowsA | Control and arrange multiple macOS windows in one click: apply automatic multi-window layouts (tile_left_right, tile_grid, quadrants, cascade, maximize, center, left_half, right_half, top_half, bottom_half) or batch explicit move/resize/minimize/unminimize/raise/activate/close operations. |
| get_window_stateA | Capture the indexed Accessibility (AX) UI tree ([1], [2], ...) of any macOS window or application in the background with 1:1 window-relative @(x,y WxH) coordinates, smart diffing, and optional isolated window screenshot. |
| window_actionA | Perform an action on a macOS window or app (click, hover, move, set_value, type_text, press_key, paste with automatic clipboard restore, scroll, drag, select_text, perform_secondary_action) with smooth 60fps virtual cursor feedback. Element-index actions operate in the background without moving the user's physical cursor. |
| browser_listA | List all available browsers (live CDP browsers, managed background 'iab' Chromium, Safari/Chrome/Brave/Edge) and their open tabs. |
| browser_open_tabA | Open one or multiple controllable browser tabs in the background (uses live Antigravity Extension Chrome Tab Group when connected, or auto-connects to CDP / auto-launches Chromium, or opens Safari tabs when browser_id='safari') and return their indexed DOM/AX trees. |
| browser_tab_stateA | Inspect a browser tab's indexed DOM/Accessibility tree ([1], [2], ...) and/or capture a live background JPEG screenshot via CDP (works even when minimized or occluded, with zero Screen Recording permission required). |
| browser_tab_actionC | Operate any browser tab in the background via Extension CDP, raw CDP, or AppleEvents without stealing focus (with live 0.72x in-tab virtual cursor, click ripples, and element highlight): click, hover, move, set_value, type_text, press_key, paste (text/md/html without touching OS clipboard), select_text, perform_secondary_action, scroll, drag, goto, back, forward, reload, close, evaluate, upload_files (Two-Tier CDP + DataTransfer fallback), wait_for_download, list_downloads, get_dialog, handle_dialog, get_console_logs, get_network_requests, get_network_body, get_cookies, set_cookies, clear_cookies, get_storage, set_storage, wait_for. |
| browser_manageB | Manage browser sessions, Chrome Tab Groups, tab ownership/lifecycle, history, and bookmarks via the Antigravity Browser Extension bridge: status, list_tab_groups, create_tab_group, group_tabs, update_tab_group, name_session, ungroup_tabs, claim_tab (in-place without moving user's tab), mark_tab ('deliverable' | 'handoff' | 'scratch'), finalize_session (closes scratch tabs + detaches debuggers so Chrome/Brave banner vanishes), search_history, list_bookmarks, create_bookmark. |
| cursor_controlA | Inspect, move, hover, highlight, hide, clear, or configure the Codex-style visual cursor & element bounding-box overlays across macOS windows (native 60fps AppKit overlay windows) and browser tabs (ShadowRoot CSS overlays). |
| batch_executeB | Execute multiple operations across ANY combination of macOS windows, window layouts, app launches, virtual cursors, and multiple browser tabs in ONE single click/call (parallel by default). |
| jsA | Codex-identical persistent JavaScript REPL (cua_repl). Top-level bindings persist across calls. Exposes |
| js_resetA | Reset the persistent JavaScript REPL context ( |
| permissionsB | Check or request macOS Accessibility and Screen Recording TCC permissions. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 17 tools
Most tools target distinct domains (apps, windows, browser tabs, cursor, permissions), but there are overlapping entry points: get_state duplicates app/window/browser listing, browser_manage broadly overlaps the browser_* family, and batch_execute vs js both serve multi-step orchestration. Descriptions help, but an agent can reasonably hesitate between these alternatives.
Names are consistently snake_case and grouped by prefix (browser_*, window-related), but the convention is mixed: some are verb_noun (list_apps, get_window_state) while others are noun_verb/noun phrases (window_action, browser_tab_state, cursor_control, batch_execute, browser_manage) and a few are bare nouns (permissions, js). Readable, but not a predictable verb_noun pattern.
17 tools is at the heavy end of the 3-15 guideline, and the surface includes broad mega-tools (browser_manage, window_action, browser_tab_action) alongside overlapping orchestration tools (batch_execute, js). Given the broad computer-use scope it is defensible, but it feels overloaded rather than tightly scoped.
Coverage is strong across macOS apps/windows, browsers, cursor overlays, batching, scripting, and permissions, with lifecycle management for windows and browser tabs. Minor gaps remain: no explicit app quit/terminate operation, and global system-level controls are absent, but agents can work around these via window/browser close or js.