native-cua

Observe and control macOS apps through a CLI or MCP server, with a floating live preview and silent MP4 recording. The independent Swift backend uses public Apple APIs; no Codex installation or OpenAI credentials are required.

Apple Silicon · macOS 14+ · Node.js 22+. Live preview and recording require macOS 15+. This is an accessibility-based prototype, not universal mouse automation.

Install

v0.4.0 is Developer ID–signed, notarized by Apple, and stapled. No Git, Xcode, or source build is needed.

1. Install the release. You can inspect the installer before running it.

   curl -fsSL https://raw.githubusercontent.com/R44VC0RP/native-cua/v0.4.0/install.sh | sh

2. Open permission setup.

   "$HOME/.local/bin/native-cua" setup

   Click Allow for Accessibility and Screen recording, then approve Native CUA in System Settings. If the app is missing from the list, drag the app tile beside Settings into it. If access still appears missing after enabling it, use Restart helper; restart is blocked during a command or capture and invalidates old snapshots.

3. Connect your MCP client. For OpenCode 2:

   opencode2 mcp add native_cua --global -- "$HOME/.local/bin/native-cua" mcp

   For other clients, use the absolute CLI path printed by the installer as the command, with mcp as its argument.

4. Ask your agent to call status. Confirm the required permissions are granted and execution_host.kind is launchservices-app.

The setup window checks the exact app helper used by that installation's CLI and MCP. Opening the app directly also shows setup, but an unattached window checks only the app's grants—not an MCP connection. Unlike earlier releases, v0.4.0 owns its permissions instead of inheriting Terminal or Node's grants.

Only clicking Allow requests access. Status checks and background startup never prompt or open setup. No microphone or Full Disk Access permission is requested, and the app does not read or modify the TCC database.

Installation details

The default paths are ~/.local/bin/native-cua and ~/.local/share/native-cua/Native CUA.app. The installer does not change your PATH, shell configuration, MCP configuration, or macOS permissions. Examples below use native-cua; if it is not on your PATH, use the full CLI path printed by the installer.

Existing apps and launchers are never overwritten. For an upgrade, review the existing installation or choose a fresh location with sh install.sh --prefix /absolute/new-prefix. There is no force-overwrite option.

The installer verifies the SHA-256 checksum, bundle identity, Ryan Vogel's Apple-issued Developer ID (9G68SMNHEU), and Gatekeeper's notarized assessment. It never removes quarantine or disables macOS security.

Related MCP server: Automation MCP

Observe and control

CLI output is JSON. Discover the app and window, then observe the exact target. Replace PID and WINDOW_ID with values from the preceding output; the screenshot destination must be a new file.

native-cua apps
native-cua windows --pid PID
native-cua state --pid PID --window WINDOW_ID --session demo --image ./window.png

Use the returned snapshot and element IDs for an action:

native-cua click --snapshot SNAPSHOT_ID --element ELEMENT_ID --session demo

Observe → act → observe. Each action consumes its snapshot; snapshots also expire after 60 seconds. Obtain fresh state to verify the result and before another action. An accepted dispatch—or an error after dispatch—is not proof of the outcome. Never blindly retry.

Keep the same --session across related commands. MCP clients can use _meta.sessionID or NATIVE_CUA_SESSION; without either, each MCP connection gets an isolated owner. These IDs separate owners, not authentication—the private local socket is the security boundary.

Use native-cua help for commands and argument schemas. The shared tool definitions describe all 14 MCP tools; setup is a user-facing CLI command, not a model tool.

Live preview and recording

Requires macOS 15+. Use a PID and window ID from a fresh observation, and keep the same session throughout the capture.

1. Start a floating preview of that window.

   native-cua capture-start --pid PID --window WINDOW_ID --session demo

2. Add recording using the returned CAPTURE_ID. The absolute destination directory must exist, and the .mp4 file must not exist.

   native-cua capture-update --capture CAPTURE_ID --recording-path /absolute/new.mp4 --session demo

3. Check capture and recording state.

   native-cua capture-status --capture CAPTURE_ID --session demo

4. Stop the stream and finalize the recording.

   native-cua capture-stop --capture CAPTURE_ID --session demo

To start preview and recording together, add --recording-path to capture-start; add --preview false for recording only. With capture-update, --preview false hides the preview without stopping an active recording, while --stop-recording true finalizes the video and leaves an enabled preview running.

One exact-window stream feeds both outputs. Resizing or hiding the preview does not change recording resolution. The preview does not activate the target or move the cursor, and is a viewer—not an input surface. Capture excludes audio, the desktop, the real cursor, and separately composited child windows/dialogs.

Defaults are 15 requested FPS, a 15-minute automatic stop, and a maximum video dimension of 1920 pixels. --fps accepts 1–30; --max-duration-seconds accepts 1–3600. Only one capture can run per daemon. It stops if the target disappears or its time limit expires; closing the CLI or MCP connection alone does not stop it.

Recordings are silent H.264 MP4 files, staged privately and published with 0600 permissions only after finalization. Existing destinations are never overwritten, even if created during recording. Check recording.complete before using the file. Forced termination can leave a private partial recording; if stopping the capture producer cannot be confirmed, the helper exits and invalidates its snapshots rather than continuing invisibly.

Limits and verification

• Clicks use declared accessibility actions (AXPress or explicit AXOpen), not raw mouse injection. Canvas clicks, right/middle clicks, hover, drag, wheel gestures, and background keyboard input are unsupported.

• Accessibility clicks, field setting, and supported scrolling do not activate the target by default. Keyboard input requires verified foreground focus unless --focus true explicitly permits activation. Coordinate clicks use a fresh window screenshot, not desktop coordinates.

• Scrolling supports an exact writable scrollbar's normalized position (0..1) or declared page actions. Apps may reject those actions; there is no automatic pointer fallback or unit conversion.

• Secure-field input is refused, but screenshot and video pixels are not redacted. Capture only authorized content and stop before displaying secrets. Permission grants do not authorize unrelated tasks or bypass authentication.

These counts mean at least one verified real-host example per tool family, not code coverage, feature completeness, or reliability across applications. Verification combines earlier control/capture checks with the v0.4.0 app-host and permission flow. It used one Apple Silicon Mac on macOS 27 beta and a limited set of apps; other macOS versions, Intel, fresh-OS onboarding, and the manual drag-to-add gesture remain unverified. See the release notes for the latest checks.

Development

Run from source

Requires Node.js 22+ and Xcode command-line tools with the required Apple SDKs. There are no npm dependencies.

1. Clone the repository.

   git clone https://github.com/R44VC0RP/native-cua.git
   cd native-cua

2. Build the native driver.

   node scripts/build.mjs

3. Check the invoking host's permissions without prompting.

   node src/cli.mjs status

Source builds produce .build/native-cua-driver and retain the invoking host's permission ownership. Grant missing access to that host in System Settings; setup intentionally requires an installed app. For source-mode MCP, use node with the absolute path to src/mcp.mjs.

After rebuilding, stop any live capture, then run node src/cli.mjs daemon stop so the next call loads the new driver. Stopping invalidates all snapshots. Idle shutdown allows bounded recording finalization; interrupting a dispatched request terminates immediately. Do not delete socket or lock files blindly after a crash.

Build a signed release

Requires the publisher's Developer ID Application identity and a notarization Keychain profile.

node scripts/package-release.mjs --identity DEVELOPER_ID_SHA1 --notary-profile PROFILE_NAME

The packager seals the arm64 app and its CLI/MCP resources, notarizes and staples it, and requires Gatekeeper acceptance before producing the ZIP, SHA256SUMS, and release.json. Output directories must be new. To resume a pending submission, use the recorded ID with --resume and the original --output directory; do not rebuild or upload again. The installer accepts paired --archive and --checksums paths for local archives, with the same trust checks.

Code layout

The native boundary is serial JSONL: a request such as {"id":1,"method":"status","params":{},"owner":"example-session"} receives a matching result or error. The macOS app host also emits a private setup_restart_requested event for the daemon. See Protocol.swift and Driver.swift. The current Unix-socket/POSIX transport is not a Windows implementation; future backends must preserve permission, targeting, ownership, and no-retry semantics without silently substituting another engine.