OpenCU
OpenCU is a local MCP server that lets an AI agent read and operate desktop apps on macOS, Windows, Linux, and Android using accessibility state, screenshots, and synthetic input.
Discover running/launchable apps and read a target app’s accessibility tree, optionally with a screenshot (
list_apps,get_app_state).Click, drag, scroll, type Unicode text, paste text/Markdown/HTML, press keys/shortcuts, set values, select text (macOS/Linux only), and invoke exposed accessibility actions.
Bring an app to the foreground when needed (
activate_app) and use screenshot coordinates for visual content.Run batches of up to 16 known actions with waits and return the final state (
run_actions).Wait for expected UI text and rely on fresh observations before input; coordinate input requires a fresh screenshot.
Respect platform constraints and user control: foreground limits, locked-screen refusal, and person-triggered stops that end the connection.
Provides desktop computer-use automation on Linux X11, allowing agents to inspect and operate local app windows through accessibility trees, screenshots, clicks, typing, scrolling, and keyboard shortcuts.
Provides computer-use automation on macOS, enabling agents to read app accessibility trees, capture screenshots, and perform clicks, typing, key presses, and other UI actions in local apps, with optional agent desktop on a virtual display.
https://github.com/user-attachments/assets/5da04b21-47fe-44c4-9d3d-e2b3a2e9a9c6
Why OpenCU
It works beside you. On macOS, input goes to the target app's process and window. Your cursor stays where it is and the app in front keeps the keyboard.
It can have a screen of its own. Start the agent desktop and the agent works on a virtual display (macOS) or in a separate session (Windows), which you watch in an ordinary window.
You stay in charge. You can always stop it: the Stop button on macOS, a physical Escape on Windows and Linux, three presses of volume-down on Android. On Windows, Linux and Android it will not take the foreground within 10 seconds of your last input. A stopped connection never restarts on its own.
It is measured. On Windows tasks that can only be done through the UI, the same agent passed 7 of 8 trials with OpenCU and 2 of 8 with a shell and screenshots. Details below.
It is yours. MIT licensed, with no account, no proprietary runtime and nothing hosted. The agent you already use brings the model.
Related MCP server: open-computer-use
Install
Windows · PowerShell, as your normal user, no administrator rights
irm https://github.com/Daily-AC/opencu/releases/latest/download/install.ps1 | iexLinux · x86_64 with glibc, X11 desktops
curl -fsSL https://github.com/Daily-AC/opencu/releases/latest/download/install.sh | shmacOS · Apple silicon, macOS 14 or later, Node.js 22+
npm install -g @z10/opencu
opencu setup # grant Accessibility and Screen RecordingAndroid · preview, Android 14 or later, driven from a computer over adb (USB or wireless debugging)
npm install -g @z10/opencu # Apple silicon Macs, with adb; other computers build from source, see the Android guide
opencu android doctorTo check an installation, run opencu doctor (on Windows and Linux, the command from the table below). Platform guides:
macOS · Windows · Linux · Android. Connecting your own host to Android: Android integration.
Connect your agent
OpenCU is a stdio MCP server. The command to register is:
Platform | Command |
macOS |
|
Windows |
|
Linux |
|
Android |
|
# Claude Code
claude mcp add --scope user opencu -- opencu
# Codex
codex mcp add opencu -- opencuOn Windows, Linux and Android, use the command from the table instead of opencu.
Any other MCP client (on Android, add "args": ["android"]):
{
"mcpServers": {
"opencu": { "command": "opencu" }
}
}Then ask your agent to do something in an app. It starts by reading the app with
get_app_state.
What the agent can do
Tool | What it does |
| List running and launchable apps |
| Read a window's accessibility tree, with a screenshot on request; launches the app if needed |
| Act on an element by its index, or on screenshot coordinates |
| Type Unicode text, paste, send keys and shortcuts |
| Change values and use an element's own actions |
| Bring an app to the front when a step needs it |
| Run up to 16 actions in one call and return one final state |
| Start, check or stop the agent desktop |
select_text is macOS and Linux only. Linux has no agent_desktop.
Input requires a fresh observation of the same app, and every error starts with a
stable code such as STOPPED_BY_PERSON. Details are in Tools and
the engine contract.
Measured
With and without OpenCU. The same agent and model, with a shell and screenshots alone or with OpenCU added:
Platform | Tasks | Shell and screenshots | With OpenCU |
Windows 11 VM | 4 tasks that need the UI: a form, an installer, Save As, an interrupting popup | 2/8 | 7/8 |
Windows 11 VM | 3 Office and OBS tasks that can be scripted around | 6/6 | 3/6 |
Linux XFCE container | 5 desktop tasks | 10/10 | 10/10, 40–65% faster inside app windows |
On the scriptable tasks the shell-only agent wrote LibreOffice and OBS scripts instead of using the UI; two of OpenCU's three failures there were a modal-dialog defect fixed in 0.3.2. Evidence: Windows, Linux.
Against open-computer-use. The same agents, models and tasks with OpenCU and with iFurySt/open-computer-use 1.1.2 at its default settings:
Platform | Tasks | OpenCU | open-computer-use |
Linux | 5 desktop tasks, 2 trials each | 10/10 | 9/10 |
Windows 11 VM | 4 UI-only tasks, 2 trials each | 7/8 | 4/8 |
macOS VM (MacArena subset) | 10 tasks, 1 trial each | 4/10 | 4/10, the same tasks |
Linux: the two Firefox tasks took OpenCU 244 and 96 seconds (median) and open-computer-use 872 and 552. open-computer-use did not list Firefox until the agent turned on the AT-SPI bus itself, and in one trial its
set_valueandtype_textleft the Firefox entries empty without an error.Windows: in all eight open-computer-use trials the agent also wrote its own Win32 input code, and its four passes completed input that way. It wrote none with OpenCU. open-computer-use ran behind a bridge on the VM (1.5 to 7 seconds per call), so its Windows timings are not comparable.
macOS: both passed the same four tasks. On those, the agent took 544 seconds in total with OpenCU and 1,498 with open-computer-use; median click time was 292 ms against 2,029 ms. The OpenCU build in that VM was 0.2.0.
The samples are small and each platform used one model. We did not test open-computer-use's locked-screen mode, its background input on covered Chrome windows, or its global pointer setting (evidence).
Platforms
macOS | Windows | Linux | Android | |
Install | npm, Apple silicon | Prebuilt x64, unsigned | Prebuilt x64 | npm on Apple silicon Macs, otherwise build from source; runs over adb |
Input | Background, to the target app only | Foreground, after 10 s without your input | Foreground, after 10 s without your input | Foreground, after 10 s without your input |
Agent desktop | Virtual display with a watch-only preview | Separate session in a window | Not available | Virtual display; watch-only preview with the optional phone app |
Screen locked | Refused; the agent desktop pauses | Refused; the agent desktop keeps indexed clicks and values | Refused | Refused |
Stopping it
macOS: the V in the menu bar has Stop. In agent desktop mode, the preview's Stop button does the same.
Windows and Linux: a physical Escape stops input for the rest of the connection, and your own input during an action stops that action. The Windows agent desktop also has a Stop button.
Android: press volume-down three times within 1.5 seconds. While the agent works on its background screen, the optional phone app's status island also has Stop. Touching the main screen during an action there stops that action.
A person's stop is final for the connection. Agents must never reconnect around it.
OpenCU is a person-facing brake, not a sandbox: the agent's apps share your account
and files, and any local process that can run opencu uses its permissions.
Limitations
macOS: Apple silicon only. The package is signed with OpenCU's own certificate and is not notarized, so install it with npm, not a browser download. macOS asks about once a month whether OpenCU may keep capturing the screen. It cannot operate a locked Mac.
Windows: the package is unsigned, so SmartScreen or Defender may warn. Outside the agent desktop, input uses the foreground.
Linux: X11 only; Wayland is refused. Input uses the foreground.
Android: a preview, tested on one phone (OPPO PGBM10, Android 14). It runs from a computer over adb; the npm package installs only on Apple silicon Macs, and other computers build from source. The phone app is optional and only shows status, a preview and Stop.
Custom-drawn apps such as Blender, and windows in the Stage Manager strip, need the foreground.
The full list, with measurements, is in Limitations.
Documentation
Tools and the engine contract, with the Windows and Linux contracts
Host integration, for programs that start OpenCU on an agent's behalf
Development
npm test
python3 scripts/test-native.py
node scripts/verify-gui.mjsverify-gui.mjs builds two disposable AppKit fixtures, keeps one in the
foreground and operates the other in the background through the MCP server. On
Windows, node scripts/verify-gui-win.mjs uses disposable WinForms and Chrome
fixtures and takes the foreground.
python3 scripts/package-opencu.py builds a self-contained archive with Node and
the MCP dependencies for your own Macs, signed with your local identity, and
python3 scripts/verify-release.py checks it offline.
License
Original OpenCU code is available under the MIT License. Bundled open source dependencies retain their own licenses. No license for proprietary third party software is included or implied.
Available Tools
13 toolsactivate_appA
Bring the requested app forward and return its state. Use when foreground interaction is required. On Windows and Linux, changing foreground requires 10 seconds without human keyboard or mouse activity; otherwise PERSON_ACTIVE is returned with no foreground change or input. Inspect the returned window before input and do not compete with user activity.
Inputs: app (required), disableDiff, waitForText, waitTimeoutMs, +1 more.
clickA
Click an observed control, including buttons and checkboxes, or a fresh screenshot coordinate. In a locked Windows agent desktop, use a single left indexed click without modifiers or duration. Returns updated state: indexed clicks return text; coordinate clicks also return a new screenshot for the next action. On macOS, clicking a menu bar item or submenu lists its items without displaying a menu; clicking a listed menu item runs its command.
Inputs: app (required), x, y, key, +5 more.
dragC
Drag inside the fresh app-window screenshot and return a new screenshot and state. To crop a side while preserving height: zoom out for margins, Select All, then move only the middle left/right selection handle horizontally. Keep the existing top/bottom selection edges; verify dimensions in Inspector. Window borders resize the window.
Inputs: app (required), to_x (required), to_y (required), from_x (required), +2 more.
get_app_stateARead-only
Start here: read the requested app by name or platform-specific identifier; it launches if needed. A locked Windows agent desktop can launch without OpenCU activating the window. Windows elevation-required launches return LAUNCH_FAILED with guidance for the person; OpenCU never elevates the program. Text normally suffices for menus, dialogs, indexed controls and text editing. Request an image for visual content or coordinates. For separately saved edits, create and verify the destination copy before editing because apps may autosave the source. Verify the saved result and any required source preservation before reporting completion; an action success only confirms input was sent.
Inputs: app (required), scope, disableDiff, waitForText, +2 more.
list_appsARead-only
Discover app names and running state when the target is unknown. If the task names an app, start with get_app_state directly.
Inputs: no named parameters.
pasteA
Paste text, Markdown, or HTML into the focused field (Windows and Linux: plain text only). Prefer this for Unicode or multiline text. Verify focus and the resulting value. Restores the previous clipboard unless another app changed it during the operation. Returns the updated accessibility state after the action; a separate text-only get_app_state is normally unnecessary. Request a new screenshot before later coordinate input.
Inputs: app (required), text (required), format (required), scope.
perform_secondary_actionA
Perform an accessibility action exposed by the latest app state. Do not invent action names. A locked-parent Windows agent desktop calls the listed pattern directly without pointer fallback. On macOS, AXShowMenu and AXScrollToVisible are omitted from listings because nearly every element supports them. Returns the updated accessibility state after the action; a separate text-only get_app_state is normally unnecessary. Request a new screenshot before later coordinate input.
Inputs: app (required), action (required), element_index (required), scope.
press_keyA
Send a key or key combination to the app, using xdotool-style syntax such as Return or ctrl+a. super is Command on macOS, the Windows key on Windows and Super on Linux. Windows holds an omitted duration for 50 ms so frame-polled games can detect taps. If state warns of native IME mode, letter keys may reach the input method; type_text and paste are unaffected. Returns the updated accessibility state after the action; a separate text-only get_app_state is normally unnecessary. Request a new screenshot before later coordinate input.
Inputs: app (required), key (required), scope, duration.
run_actionsA
Run a short known sequence and return its final state. Single-action tools already return text state; use this for several known actions or an inline final screenshot. Example: focus an observed field, select all, paste, then inspect the result. Stop when new information or approval is needed. Accepts {"method":"wait","ms":N} for 1–15000 ms on every platform; wait sends no input and honours person stops. Coordinate input must be first (including before any wait) and use a fresh screenshot. Stops on error or an unknown accessibility action outcome without replay; earlier actions may have run. An unknown outcome requires an explicit get_app_state before continuing. A locked pattern that refreshes indexes stops the remaining batch and returns fresh state; choose subsequent actions from that state.
Inputs: app (required), actions (required), scope, disableDiff, +3 more.
scrollA
Scroll an observed element or fresh screenshot coordinate. Returns current state: text for indexed input, plus a new screenshot for coordinate input.
Inputs: app (required), direction (required), x, y, +3 more.
select_textA
Not yet supported on Windows. Select matching text, or place the cursor beside it, in an observed editable element. Returns the updated accessibility state after the action; a separate text-only get_app_state is normally unnecessary. Request a new screenshot before later coordinate input.
Inputs: app (required), text (required), element_index (required), scope, +3 more.
set_valueA
Set a value only on an element explicitly marked settable in the current tree, then return app state. Use click for buttons and checkboxes. In a locked-parent Windows agent desktop this requires ValuePattern; RangeValue is not available there. Verify the resulting value; web application input handlers may behave differently from typing or pasting. Returns the updated accessibility state after the action; a separate text-only get_app_state is normally unnecessary. Request a new screenshot before later coordinate input.
Inputs: app (required), value (required), element_index (required), scope.
type_textA
Type Unicode text into the observed app. Newlines may submit forms; use paste for multiline content. Verify the resulting value. Returns the updated accessibility state after the action; a separate text-only get_app_state is normally unnecessary. Request a new screenshot before later coordinate input.
Inputs: app (required), text (required), scope.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 13 tools
Most tools target distinct actions (click, drag, scroll, press_key vs. state reading), and descriptions clarify boundaries such as using click for checkboxes versus set_value only on settable elements. Some overlap remains among input paths (type_text vs. paste vs. set_value) and between get_app_state and activate_app, but guidance mostly resolves it.
All names are snake_case and action-oriented, with a dominant verb_noun pattern (get_app_state, run_actions, press_key, set_value, perform_secondary_action). Minor deviations are bare verbs like click, drag, paste, and scroll, which are still unambiguous and readable.
Thirteen tools is well within the ideal range for a computer-use automation server, and each covers a distinct primitive (state, click, typing, keys, scrolling, dragging, batching, app discovery/activation). No tool feels redundant or bloated.
The surface covers the core lifecycle: discover/launch/read app state, foreground activation, all major input modalities, batching with waits, and accessibility actions. Minor gaps exist, such as select_text being unsupported on Windows and no standalone screenshot-only or clipboard-read tools, but get_app_state and run_actions provide workarounds.
Maintenance
Related MCP Connectors
Give your AI hands on every computer you own. Remote control of your Windows, Mac and Linux machines from one ChatGPT, Claude or Cursor chat — even from your phone. Ask it to find out why the office server is crawling, update every PC in the house, sort out your mom's laptop from another city, or ship the site from your work machine while you're on a train. It runs commands, reads files and takes a screenshot when it needs to see the screen. You decide what each computer allows, and you can cut any of them off in a second.
Connect AI assistants to your approved computer for files, commands, browser and desktop tasks.
Operate Linux, macOS and Windows from your LLM. Every action runs through an auditable allowlist.
Use your Mac, Windows or Linux computer from ChatGPT, Claude or Codex: files, commands, documents.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables AI agents to see and control your desktop with tools for screenshots, clicks, typing, and more, all locally on macOS and Windows.107 npmMIT
- AlicenseNot gradedqualityBmaintenanceLets AI agents see and control desktop applications through the accessibility layer, enabling clicking, typing, scrolling, dragging, and window/app management across macOS, Windows, and Linux entirely on the local machine.20MIT
- AlicenseAqualityAmaintenanceAn open-source MCP server that lets any coding agent operate your computer like a person does—reading the UI through accessibility trees, clicking and typing in the background, showing an agent pointer, zooming into regions, and driving your signed-in Chrome on macOS, Windows, and Linux.3735Apache 2.0
- AlicenseNot gradedqualityCmaintenanceEnables AI models to control a local Windows desktop by moving the cursor, clicking, scrolling, typing Unicode text, sending key chords, and capturing screenshots. Runs natively on Windows with minimal latency and no heavy dependencies such as Python, PyAutoGUI, or Docker.119 npm1MIT