# native-cua

Observe and control macOS apps through a CLI or MCP server, with a floating live preview and silent MP4 recording. The independent Swift backend uses public Apple APIs; no Codex installation or OpenAI credentials are required.

**Apple Silicon · macOS 14+ · Node.js 22+**. Live preview and recording require macOS 15+. This is an accessibility-based prototype, not universal mouse automation.

## Install

[**v0.4.0**](https://github.com/R44VC0RP/native-cua/releases/tag/v0.4.0) is Developer ID–signed, notarized by Apple, and stapled. No Git, Xcode, or source build is needed.

1. Install the release. You can [inspect the installer](https://github.com/R44VC0RP/native-cua/blob/v0.4.0/install.sh) before running it.

   ```sh
   curl -fsSL https://raw.githubusercontent.com/R44VC0RP/native-cua/v0.4.0/install.sh | sh
   ```

2. Open permission setup.

   ```sh
   "$HOME/.local/bin/native-cua" setup
   ```

   Click **Allow** for Accessibility and Screen recording, then approve **Native CUA** in System Settings. If the app is missing from the list, drag the app tile beside Settings into it. If access still appears missing after enabling it, use **Restart helper**; restart is blocked during a command or capture and invalidates old snapshots.

3. Connect your MCP client. For OpenCode 2:

   ```sh
   opencode2 mcp add native_cua --global -- "$HOME/.local/bin/native-cua" mcp
   ```

   For other clients, use the absolute CLI path printed by the installer as the command, with `mcp` as its argument.

4. Ask your agent to call `status`. Confirm the required permissions are granted and `execution_host.kind` is `launchservices-app`.

The setup window checks the exact app helper used by that installation's CLI and MCP. Opening the app directly also shows setup, but an unattached window checks only the app's grants—not an MCP connection. Unlike earlier releases, v0.4.0 owns its permissions instead of inheriting Terminal or Node's grants.

Only clicking **Allow** requests access. Status checks and background startup never prompt or open setup. No microphone or Full Disk Access permission is requested, and the app does not read or modify the TCC database.

### Installation details

The default paths are `~/.local/bin/native-cua` and `~/.local/share/native-cua/Native CUA.app`. The installer does not change your PATH, shell configuration, MCP configuration, or macOS permissions. Examples below use `native-cua`; if it is not on your PATH, use the full CLI path printed by the installer.

**Existing apps and launchers are never overwritten.** For an upgrade, review the existing installation or choose a fresh location with `sh install.sh --prefix /absolute/new-prefix`. There is no force-overwrite option.

The installer verifies the SHA-256 checksum, bundle identity, Ryan Vogel's Apple-issued Developer ID (`9G68SMNHEU`), and Gatekeeper's notarized assessment. It never removes quarantine or disables macOS security.

## Observe and control

CLI output is JSON. Discover the app and window, then observe the exact target. Replace `PID` and `WINDOW_ID` with values from the preceding output; the screenshot destination must be a new file.

```sh
native-cua apps
native-cua windows --pid PID
native-cua state --pid PID --window WINDOW_ID --session demo --image ./window.png
```

Use the returned snapshot and element IDs for an action:

```sh
native-cua click --snapshot SNAPSHOT_ID --element ELEMENT_ID --session demo
```

**Observe → act → observe.** Each action consumes its snapshot; snapshots also expire after 60 seconds. Obtain fresh state to verify the result and before another action. An accepted dispatch—or an error after dispatch—is not proof of the outcome. Never blindly retry.

Keep the same `--session` across related commands. MCP clients can use `_meta.sessionID` or `NATIVE_CUA_SESSION`; without either, each MCP connection gets an isolated owner. These IDs separate owners, not authentication—the private local socket is the security boundary.

Use `native-cua help` for commands and argument schemas. The [shared tool definitions](src/tools.mjs) describe all 14 MCP tools; `setup` is a user-facing CLI command, not a model tool.

## Live preview and recording

Requires macOS 15+. Use a PID and window ID from a fresh observation, and keep the same session throughout the capture.

1. Start a floating preview of that window.

   ```sh
   native-cua capture-start --pid PID --window WINDOW_ID --session demo
   ```

2. Add recording using the returned `CAPTURE_ID`. The absolute destination directory must exist, and the `.mp4` file must not exist.

   ```sh
   native-cua capture-update --capture CAPTURE_ID --recording-path /absolute/new.mp4 --session demo
   ```

3. Check capture and recording state.

   ```sh
   native-cua capture-status --capture CAPTURE_ID --session demo
   ```

4. Stop the stream and finalize the recording.

   ```sh
   native-cua capture-stop --capture CAPTURE_ID --session demo
   ```

To start preview and recording together, add `--recording-path` to `capture-start`; add `--preview false` for recording only. With `capture-update`, `--preview false` hides the preview without stopping an active recording, while `--stop-recording true` finalizes the video and leaves an enabled preview running.

One exact-window stream feeds both outputs. Resizing or hiding the preview does not change recording resolution. The preview does not activate the target or move the cursor, and is a viewer—not an input surface. Capture excludes audio, the desktop, the real cursor, and separately composited child windows/dialogs.

Defaults are **15 requested FPS**, a **15-minute automatic stop**, and a maximum video dimension of **1920 pixels**. `--fps` accepts 1–30; `--max-duration-seconds` accepts 1–3600. Only one capture can run per daemon. It stops if the target disappears or its time limit expires; closing the CLI or MCP connection alone does not stop it.

Recordings are silent H.264 MP4 files, staged privately and published with `0600` permissions only after finalization. Existing destinations are never overwritten, even if created during recording. Check `recording.complete` before using the file. Forced termination can leave a private partial recording; if stopping the capture producer cannot be confirmed, the helper exits and invalidates its snapshots rather than continuing invisibly.

## Limits and verification

- Clicks use declared accessibility actions (`AXPress` or explicit `AXOpen`), not raw mouse injection. Canvas clicks, right/middle clicks, hover, drag, wheel gestures, and background keyboard input are unsupported.
- Accessibility clicks, field setting, and supported scrolling do not activate the target by default. Keyboard input requires verified foreground focus unless `--focus true` explicitly permits activation. Coordinate clicks use a fresh window screenshot, not desktop coordinates.
- Scrolling supports an exact writable scrollbar's normalized position (`0..1`) or declared page actions. Apps may reject those actions; there is no automatic pointer fallback or unit conversion.
- Secure-field input is refused, but screenshot and video pixels are **not redacted**. Capture only authorized content and stop before displaying secrets. Permission grants do not authorize unrelated tasks or bypass authentication.

| Platform | Status | Verified tool families |
| --- | --- | --- |
| macOS | Apple Silicon prototype | 14/14 |
| Windows | Not implemented | 0/14 |
| Linux | Not implemented | 0/14 |

These counts mean **at least one verified real-host example per tool family**, not code coverage, feature completeness, or reliability across applications. Verification combines earlier control/capture checks with the v0.4.0 app-host and permission flow. It used one Apple Silicon Mac on macOS 27 beta and a limited set of apps; other macOS versions, Intel, fresh-OS onboarding, and the manual drag-to-add gesture remain unverified. See the [release notes](https://github.com/R44VC0RP/native-cua/releases/tag/v0.4.0) for the latest checks.

## Development

### Run from source

Requires Node.js 22+ and Xcode command-line tools with the required Apple SDKs. There are no npm dependencies.

1. Clone the repository.

   ```sh
   git clone https://github.com/R44VC0RP/native-cua.git
   cd native-cua
   ```

2. Build the native driver.

   ```sh
   node scripts/build.mjs
   ```

3. Check the invoking host's permissions without prompting.

   ```sh
   node src/cli.mjs status
   ```

Source builds produce `.build/native-cua-driver` and retain the invoking host's permission ownership. Grant missing access to that host in System Settings; `setup` intentionally requires an installed app. For source-mode MCP, use `node` with the absolute path to `src/mcp.mjs`.

After rebuilding, stop any live capture, then run `node src/cli.mjs daemon stop` so the next call loads the new driver. Stopping invalidates all snapshots. Idle shutdown allows bounded recording finalization; interrupting a dispatched request terminates immediately. Do not delete socket or lock files blindly after a crash.

### Build a signed release

Requires the publisher's Developer ID Application identity and a notarization Keychain profile.

```sh
node scripts/package-release.mjs --identity DEVELOPER_ID_SHA1 --notary-profile PROFILE_NAME
```

The packager seals the arm64 app and its CLI/MCP resources, notarizes and staples it, and requires Gatekeeper acceptance before producing the ZIP, `SHA256SUMS`, and `release.json`. Output directories must be new. To resume a pending submission, use the recorded ID with `--resume` and the original `--output` directory; do not rebuild or upload again. The installer accepts paired `--archive` and `--checksums` paths for local archives, with the same trust checks.

### Code layout

| Path | Purpose |
| --- | --- |
| [`native/macos/`](native/macos/) | Swift driver, app host, permission UI, capture, and preview |
| [`native/windows/`](native/windows/), [`native/linux/`](native/linux/) | Reserved backend directories, not implementations |
| [`src/`](src/) | CLI, MCP server, validation, daemon, and tool schemas |
| [`scripts/`](scripts/) | Native build and signed-release packaging |

The native boundary is serial JSONL: a request such as `{"id":1,"method":"status","params":{},"owner":"example-session"}` receives a matching `result` or `error`. The macOS app host also emits a private `setup_restart_requested` event for the daemon. See [`Protocol.swift`](native/macos/Protocol.swift) and [`Driver.swift`](native/macos/Driver.swift). The current Unix-socket/POSIX transport is not a Windows implementation; future backends must preserve permission, targeting, ownership, and no-retry semantics without silently substituting another engine.