Computer Use

Introduction

Computer Use is an open-source MCP server — a computer-use agent (CUA) backend — that lets any coding agent use your computer the way a person does. It reads the screen through accessibility trees, clicks and types in the background so your mouse stays yours, shows an agent pointer where it is working, zooms in on small text, and drives tabs in your signed-in Chrome — on macOS, Windows and Linux.

Works with Claude Code, Codex, Cursor and MT Code, or any other MCP client, with any model — no vision model is required for interaction.

Built by Munim Technologies as the Computer Use engine of MT Code, and published here on its own.

Related MCP server: openowl

Table of contents

• Quick start

• Capability matrix

• Why it works well

• Tools

• Repository layout

• Environment flags

• Prompting your agent

• Contributing

• Credits and license

Quick start

1. Download the latest binary for your platform from Releases (computer-use-macos-universal.zip, computer-use-windows-x64.zip) or build from source.

2. Put it somewhere on your PATH (/usr/local/bin/computer-use, or %LOCALAPPDATA%\Programs\computer-use\computer-use.exe).

3. macOS only: run computer-use request-permissions once to be prompted for Accessibility and Screen Recording.

4. Register it with your agent:

# Claude Code — fastest: the npm launcher fetches the signed binary on first run
claude mcp add computer-use -- npx -y munim-computer-use
# or point at a downloaded binary
claude mcp add computer-use -- /usr/local/bin/computer-use

# Codex — ~/.codex/config.toml
[mcp_servers.computer-use]
command = "npx"
args = ["-y", "munim-computer-use"]

// Cursor — .cursor/mcp.json
{ "mcpServers": { "computer-use": { "command": "npx", "args": ["-y", "munim-computer-use"] } } }

Then ask: "Open Safari, find the cheapest flight to Denver on Tuesday and put it in a note." The agent reads the UI with get_app_state, acts by element id, and verifies with screenshot.

Capability matrix

Munim Computer Use is the highlighted first column; the others are the computer-use servers people reach for. each cell comes from that project's own README in September 2026 (sources under Credits and license). ✅ present · ❌ absent or not documented · ⚠️ partial.

Also looked at: mediar-ai/mcp-server-macos-use (macOS, accessibility, real input), deploymenttheory/windows-mcp-server (Windows, UIA Invoke patterns, WaitFor), nuphus-mcp (OCR + bring-your-own vision model, CDP Chrome), computer-control-mcp (PyAutoGUI + OCR), and microsoft/playwright-mcp (browser only). Corrections welcome — open an issue with a link.

Why it works well

• Accessibility first, pixels second. get_app_state returns the app's accessibility tree with stable element ids, so the agent presses the button instead of guessing at a coordinate. It costs a fraction of the tokens of a screenshot and it is what scores highest on OSWorld-style tasks. Screenshots are for verifying and for content the tree cannot describe.

• Background control. Events are addressed to the target window (SkyLight on macOS, posted window messages on Windows, XTEST on Linux). No focus stealing, no hijacked mouse.

• Pointer overlay, not your pointer. A soft lavender agent pointer shows where the agent is acting. Your cursor is untouched.

• Coordinates that land. Every screenshot and zoom carries its screen origin and pixels-per-point. zoom captures any region at full physical resolution.

• Your browser, your logins. The Chrome extension gives the agent its own labelled tab group in your signed-in Chrome and never touches your tabs.

• Model-agnostic. No vision model is required for interaction; local models work too.

• Look → act → verify. hover for mouse-over menus, wait for loads, query to find a control by label without reading a whole tree.

Tools (26)

Names, argument shapes and descriptions are identical on every platform; a model that learned them on a Mac needs nothing new on Windows.

Repository layout

Build from source

# macOS
cd macos && swift build -c release
# Windows / Linux
cd windows-linux && cargo build --release

Linux notes: element actions work everywhere; coordinate clicks need an X11 or XWayland client, since native Wayland apps do not expose absolute geometry.

Chrome extension (optional)

1. chrome://extensions → Developer mode → Load unpacked → select chrome-extension/.

2. Register the native messaging host: sh chrome-extension/install.sh (macOS/Linux) or powershell -File chrome-extension/install.ps1 (Windows). Point COMPUTER_USE_PATH at the binary if it is not in the default build location.

Environment flags

Prompting your agent

Look → act → verify. get_app_state for ids, act by id, then get_app_state or screenshot again before the next step. Use zoom for small text, hover for menus that appear on mouse-over, wait after loads, keyboard shortcuts for stubborn widgets. The system-prompt text MT Code gives its agents lives in CodexDeveloperInstructions.ts and is a good starting point.

Contributing

This repository mirrors the native/ tree of munimtechnologies/mtcode, where the server is developed and shipped inside MT Code. Issues and discussions are welcome here; code changes land in mtcode first and are synced.

Credits and license

Designed and built by Munim Technologies (Munim, Inc.) for MT Code. Copyright 2026 Munim, Inc. Licensed under the Apache License 2.0; see LICENSE.

Comparison sources: Codex Computer Use · Anthropic computer-use demo · CursorTouch/Windows-MCP · CursorTouch/MacOS-MCP · QwenLM/open-computer-use · zavora-ai/computer-use-mcp · mediar-ai/mcp-server-macos-use · deploymenttheory/windows-mcp-server · nuphus-mcp · computer-control-mcp · microsoft/playwright-mcp