computer-use-turbo

Let any AI agent operate the desktop apps on your computer in the background, safely, with you in control.

Computer Use Turbo is a cross-platform computer-use toolkit developed by afaaq.io. It gives AI agents a fast, reliable way to see and operate real desktop applications on macOS, Windows and Linux. Instead of guessing from pixels alone, the agent reads each app's accessibility tree as numbered lines, sees a screenshot of the window, and acts on exact elements while you keep using your computer.

It speaks the open Model Context Protocol (MCP), so it works with any MCP-capable agent: desktop assistants, IDE agents, command-line agents and agent SDKs.

Contents

• Installation

• Highlights

• Supported platforms

• Architecture

• Components

• Tools

• Safety and privacy

• Configuration

• Development

• Troubleshooting

• License

Related MCP server: servo-mcp

Installation

Every platform needs Node.js 20 or later and Git. Installing takes three steps: get the code, build the helper for your system, then connect your agent.

1. Get the code

git clone https://github.com/afaaq-io/ComputerUseTurbo.git

cd ComputerUseTurbo

2. Build and install the helper

macOS

Requirements: macOS 14 or later and Xcode or the Command Line Tools. An Apple Development signing certificate is recommended; without one, macOS forgets the granted permissions after every rebuild.

scripts/build.sh

scripts/install.sh

Then allow Computer Use Turbo in System Settings ▸ Privacy & Security under Accessibility and Screen & System Audio Recording. The installer prints shortcuts to both panes.

Windows

Requirements: Rust with the MSVC toolchain. In PowerShell:

scripts\build.ps1

scripts\install.ps1

The helper is installed to %LOCALAPPDATA%\Programs\ComputerUseTurbo. No permission grants are needed.

Linux

Requirements: Rust, a desktop session with accessibility (AT-SPI) enabled, and the PipeWire, xkbcommon and clang development packages (on Debian and Ubuntu: libpipewire-0.3-dev libspa-0.2-dev libxkbcommon-dev clang libclang-dev). Wayland sessions need a desktop portal with RemoteDesktop and ScreenCast support (GNOME, KDE Plasma, COSMIC).

scripts/build.sh

scripts/install.sh

The helper is installed to ~/.local/bin/computer-use-turbo-helper. On Wayland, the installer asks you once to allow screen sharing. To enable accessibility on GNOME:

gsettings set org.gnome.desktop.interface toolkit-accessibility true

3. Connect your agent

The MCP server is server/src/server.mjs inside your copy. In the commands below, replace /path/to/ComputerUseTurbo with the folder you cloned into (pwd prints it).

The server is named turbo because some agent apps reserve names beginning with "computer-use" for their own built-in tools.

Claude Code

Install the plugin from the afaaq.io marketplace:

claude plugin marketplace add afaaq-io/ComputerUseTurbo

claude plugin install computer-use-turbo@afaaq-io

Or add the server from your copy directly:

claude mcp add --scope user turbo -- node /path/to/ComputerUseTurbo/server/src/server.mjs

Check it with claude mcp list, then start a new Claude Code session.

Codex

codex mcp add turbo -- node /path/to/ComputerUseTurbo/server/src/server.mjs

Or add it to ~/.codex/config.toml by hand:

[mcp_servers.turbo]
command = "node"
args = ["/path/to/ComputerUseTurbo/server/src/server.mjs"]

Any other MCP agent

Desktop assistants, IDE agents and agent SDKs take the standard MCP entry. Print it with the full paths for your machine filled in:

scripts/mcp-config.sh

On Windows, run scripts\mcp-config.ps1. Add the printed entry to your agent's MCP configuration:

{
  "mcpServers": {
    "turbo": {
      "command": "node",
      "args": ["/path/to/ComputerUseTurbo/server/src/server.mjs"]
    }
  }
}

Restart the agent and try, for example: "Use computer use to open TextEdit, write a short note and make the title bold."

Highlights

• Works with any agent. A standard MCP server over stdio. The agent's name and the app it runs in are detected at runtime; nothing is tied to one vendor.

• Works in the background. The agent operates apps behind your windows and never moves your mouse. A small live preview beside your agent's window shows what it is doing. The few steps that cannot run in the background (a menu-bar menu, a file drag, apps that draw their own interface) briefly bring the app forward only while you are idle and hand the front straight back.

• Built for models. Each app is presented as numbered lines (#12 button "Save") plus a screenshot. Later observations report only what changed, and what happened in between ("dialog opened", "value changed").

• Fast by design. find_command searches every command in an app's menus and run_command runs it without opening a menu. wait_for listens to the app's own events and returns the moment something happens, instead of polling.

• Reaches the whole desktop. Beyond regular apps, the agent can open system panels such as Control Center, Notification Center and Spotlight through their menu-bar status items.

• Safe by default. Password fields are never typed into, password managers need your confirmation, system authentication is protected, and you can stop the agent at any time. Everything else including terminals is available right away; your agent's own permission prompts stay in charge.

Supported platforms

Architecture

 AI agent (any MCP client)
      │   MCP over stdio: find_apps, observe_app, click_at, write_text, …
      ▼
 MCP server  (server/, Node.js — identical on every platform)
      │   JSON-RPC over a local socket (macOS, Linux) or named pipe (Windows)
      ▼
 Platform helper  (owns the OS permissions)
      reads the UI · captures windows · sends input · enforces every safety rule

The helper is the only component that touches your applications and the only place where safety is enforced. The server is a thin translator: it validates arguments, identifies the agent, and turns screenshots into image content.

Components

MCP server — server/

A Node.js MCP server shared by all platforms. It exposes the tools, validates every argument, starts the helper when it is not running, and deletes each screenshot as soon as it has been read.

macOS helper — helpers/macos/

Computer Use Turbo.app, a background app without a Dock icon, built as a Swift package with two targets:

• TurboCore — pure, testable logic with no UI dependencies: the protocol and framing, the tree model and its text format, element identity, the safety policy, key-chord parsing, and focus rules.

• TurboHelper — the application: socket server, accessibility reader, window capture, input synthesis, menu commands, the password-manager card, the status overlay, the live preview and the agent's on-screen pointer.

Windows and Linux helper — helpers/windows-linux/

A Rust workspace that builds one helper binary for both systems:

• core (turbo-core) — everything platform-neutral: protocol, error codes, the request pipeline, safety policy, key chords and the tree text format.

• helper (computer-use-turbo-helper) — the Windows layer (UI Automation, Win32), the Linux layer (AT-SPI, X11, Wayland portal and PipeWire), and the password-manager card, overlay and preview drawn with egui.

How each step is carried out:

Shared data — shared/

Scripts — scripts/

build and install for each platform (.sh for macOS and Linux, .ps1 for Windows), and mcp-config, which prints the entry to add to your agent's configuration.

Tools

Safety and privacy

All rules are enforced inside the helper, so no agent can bypass them.

• No permission pop-ups — apps, terminals included, are available right away. Your agent's own permission prompts decide what it may do.

• Protected apps — system authentication, system settings, the helper itself and the app or terminal the agent runs in can never be controlled.

• Password managers — the only place Computer Use Turbo asks you: a confirmation card appears once per task before the agent can use a password manager or keychain (including their menu-bar components).

• Password fields — the helper refuses to type into them or read their values.

• No blind clicks — a click or drag by position is refused if the window changed where the agent aimed since its last look and the agent did nothing in between; it looks again first.

• Stop at any time — press Stop on the overlay or Esc. The agent's session ends at once; it asks you in the chat before starting again.

• You stay in charge — if you start using the app the agent is working on, it pauses and waits for you.

• System-wide shortcuts — only shortcuts the operating system itself owns (for example, ⌘Space for Spotlight) are ever pressed outside the target app.

• Screenshots — kept only until the server has read them, and never longer than ten minutes.

• Logs — never contain typed text, field values or clipboard contents.

Configuration

Settings file

Optional settings live in settings.json in the helper's data folder:

A file named allow-protected.txt in the same folder can un-protect apps from the default policy, one app id per line. The helper itself and the agent's own app always stay protected.

Environment variables

File locations

Development

Project structure

server/                   MCP server (Node.js, all platforms)
helpers/macos/            macOS helper (Swift package → Computer Use Turbo.app)
helpers/windows-linux/    Windows and Linux helper (Rust workspace)
shared/                   Safety policy and design tokens used by every helper
scripts/                  Build, install and MCP configuration scripts

Running the tests

(cd server && npm test)

(cd helpers/macos && swift test)

(cd helpers/windows-linux && cargo test)

The tests cover the rules that keep users safe — password-field detection, the protected and sensitive app lists, system-wide shortcuts and screenshot file handling — as well as the tree text format, key parsing and the wire protocol.

Troubleshooting

• accessMissing or no screenshots (macOS) — allow Computer Use Turbo under Accessibility and Screen & System Audio Recording, then restart the helper: pkill -f "Computer Use Turbo.app/Contents/MacOS/TurboHelper".

• Permissions reset after every build (macOS) — the helper was signed ad hoc. Install an Apple Development certificate (Xcode ▸ Settings ▸ Accounts) or set CUT_SIGN_IDENTITY.

• userActive — the step needs the app in front and you were typing or using the mouse. The agent retries a moment later.

• No screenshots on Wayland — run computer-use-turbo-helper --share-screen and approve sharing in the dialog that appears.

• Logs — see File locations. The server logs to standard error, which your agent captures.

License

Computer Use Turbo is released under the MIT License.

Copyright © 2026 afaaq.io.