Skip to main content
Glama
lhwlhw0829-cmd

claude-background-control

by lhwlhw0829-cmd
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

claude-background-control

An MCP server that lets Claude Code use your Mac's GUI: read what's on screen, click things, and type. It uses only built-in macOS tools (osascript, screencapture, sips), so there are no native npm dependencies to break on install.

It goes through the accessibility tree first. That way it can press buttons and fill in text fields in background apps without moving your mouse or taking focus away from what you're doing. Real mouse and keyboard events are only the fallback.

Install

claude mcp add computer-control -- npx -y claude-background-control

Requires macOS and Node 20+ (development needs 22.18+ to run the TypeScript tests directly).

Related MCP server: Desktop Pilot MCP

Permissions (first run)

macOS has to allow the app that runs Claude Code (Terminal, iTerm, the Claude app, …):

  1. System Settings → Privacy & Security → Accessibility: turn it on for that app. Every tool except screenshot needs this.

  2. System Settings → Privacy & Security → Screen & System Audio Recording: turn it on for that app. Needed for screenshot.

  3. Restart that app.

If a permission is missing, the error names the exact app and path macOS expects. That isn't always the app you see. Claude Code inside the Claude desktop app, for example, needs the grant on ~/Library/Application Support/Claude/claude-code/<version>/claude.app, which you add with + and Cmd+Shift+G. Because that path includes the version number, a Claude Code update can require granting it again.

Tools

Tool

What it does

inspect(app?, find?)

Lists UI elements of the app's windows with an id, role, label, value, and frame. Start here. find returns only matching elements (by role or label).

click_element(id | label | role, app?)

Presses a button or focuses a field via accessibility, with no mouse movement. Falls back to a real click.

set_value(id | label | role, value, app?)

Sets a text field's value directly. Works in background apps and with any language.

type(text)

Types into the focused field by pasting, so Korean and other IME input comes through correctly. The whole clipboard (images, files, rich text) is restored afterwards.

key(combo, repeat?)

Sends key codes like cmd+s or return, so shortcuts work even when a Korean input source is active.

wait_for(find, app?, gone?, timeout?)

Waits until a matching element appears, or disappears with gone. Use after actions that open dialogs or load content.

click(x, y, button?, clicks?)

Real mouse click (CoreGraphics) for canvas or Electron content.

menu(path?, app?)

Walks the menu bar by item names, e.g. ["File", "Save…"]. A path that ends on a menu lists its items; one that ends on an item clicks it. Works on background apps.

scroll(direction, amount?, x?, y?)

Mouse-wheel scroll by lines, at a point or wherever the cursor is.

drag(x1, y1, x2, y2)

Left-button drag: select text, move things, resize.

screenshot(app? | x, y, w, h)

PNG of the main screen, a region, or one app's front window, even if other windows cover it. Image pixels equal screen points. For a window, add the reported origin to get click coordinates.

activate_app(name)

Launches an app or brings it to the front.

app is the process name (e.g. "TextEdit"). If you leave it out, the frontmost app is used. When a label or role matches in several windows, the frontmost window wins. If it's still ambiguous, you get a list of ids to choose from.

Develop

npm install
npm test        # pure-logic unit tests
npm run build

After changing src/mac.ts, run the manual GUI checklist in scripts/smoke-test.md.

Limits

  • Main display only.

  • inspect doesn't descend into leaf-like roles such as buttons and static text.

  • With Stage Manager on, a background app's window is only a side-strip thumbnail, so screenshot(app) warns and suggests activate_app first.

Design notes: docs/superpowers/specs/2026-09-30-computer-control-mcp-design.md

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • Desktop Pilot MCP

    VersoXBT
    A
    license
    Not graded
    quality
    D
    maintenance
    Enables high-speed native macOS automation for Claude by interacting directly with the Accessibility API, AppleScript, and UI trees instead of using screenshots. It allows users to read app states, click elements, and type text semantically across any macOS application.
    7 npm
    11
    MIT

  • macos-ui-mcp

    thangduonghuu
    A
    license
    A
    quality
    C
    maintenance
    Enables Claude to inspect and drive native macOS app UIs during development via an in-process view tree and screenshot renderer, without requiring screen recording permission.
    7
    MIT

  • native-cua

    R44VC0RP
    F
    license
    Not graded
    quality
    C
    maintenance
    Enables macOS computer-use automation through public Apple APIs, supporting app and window discovery, clicking, typing, scrolling, and field setting via accessibility controls.
    4
    -