mac-reader

An MCP server that reads and acts on a Mac's screen from another computer, over SSH, through the macOS Accessibility API. You get the window's structured content (roles, names, values, enabled state) as text instead of a screenshot for a vision model to guess at. Nothing is installed on the Mac.

Why

Driving a Mac from a Windows PC usually means screen sharing and screenshots. But macOS already maintains a structured tree of every window's controls for accessibility, and it can be read over a plain SSH login. It needs no remote desktop session and no app on the Mac, just Python's standard library.

It has been used for real work: driving Xcode through an archive build and a TestFlight upload from a Windows PC, reading the archive list, the run-destination picker and the distribution flow as text.

Related MCP server: gemini-mac-computer-use-mcp

How it works

Each tool call:

1. finds the Mac by its SSH host key, not by its address (find_mac.py, below);

2. sends mac_agent.py over SSH to the Mac's built-in /usr/bin/python3 -;

3. reads the JSON it prints back.

mac_agent.py calls the Accessibility API through ctypes, so there is no pyobjc and no pip on the Mac side.

Setup

On the Mac

1. Turn on Remote Login: System Settings → General → Sharing → Remote Login.

2. Allow key login from the computer you'll drive it from: add that computer's public key to ~/.ssh/authorized_keys on the Mac. Recommended: turn password login off, since this only ever uses a key. Put this in /etc/ssh/sshd_config.d/000-key-only.conf (needs sudo):

   PasswordAuthentication no
   KbdInteractiveAuthentication no

3. Grant Accessibility to /usr/libexec/sshd-keygen-wrapper: System Settings → Privacy & Security → Accessibility → +, then press ⌘⇧G and type the path. macOS runs that program for every SSH login, so this is where the permission has to go for SSH commands. Without it, every read comes back empty and the tool says so.

4. Read the Mac's host key at the Mac itself (not over the network, so you know it's the real one):

   cat /etc/ssh/ssh_host_ed25519_key.pub

On your computer

pip install -r requirements.txt
cp mac.example.json mac.json

Fill in mac.json (it's gitignored):

Check it can find the Mac:

python find_mac.py

Register it with Claude Code at user scope, so every project can use it:

claude mcp add mac-reader -s user -- python "/path/to/mac-reader/server.py"

Replace /path/to/mac-reader with wherever you cloned this repo.

Tools

Finding the Mac safely

A device is only "the Mac" if it proves it with the Mac's SSH host key. Something answering on port 22 is not identity: after the router reshuffles addresses, another device can hold the old one. So:

• every candidate is checked by ssh itself: a connection pinned to the key in mac.json runs exit 0, and any other key stops ssh before it logs in;

• every real command uses the same pinned connection (HostKeyAlias, a dedicated known_hosts file, StrictHostKeyChecking=yes), so connecting to an impostor fails instead of working;

• BatchMode=yes: it logs in with a key or not at all, and never asks for a password;

• if two devices ever present the Mac's key, it refuses to pick one;

• "presented a different key" (a security warning) and "couldn't connect" (a network problem) are reported separately, because they need different responses.

It tries the last verified address first, then scans the local network (about 3 seconds for a /24). It won't scan anything larger than 1024 addresses.

Safety rules

Each of these came from a real incident, on this tool or its Windows counterpart.

• Press at most once. AXPress is sent once. It falls back to AXPick or AXConfirm only when the app reports the action was never delivered (unsupported, invalid element).

  • kAXErrorCannotComplete does not fall through: it also comes back when the app got the press and answered slowly. Treating that as "didn't happen" is how one click becomes two.

  • An action is never retried over SSH either: the connection can drop after the command ran. Reads retry once.

• Confirm by what changed. After a press it watches the window for a second. If nothing changed, it says unconfirmed and warns not to press again.

• The real pointer is opt-in. Accessibility actions don't move it. allow_mouse=True is for controls with no action at all (Xcode's run-destination picker is one). It moves the pointer for one click and puts it back, even if the click fails.

• Never type a password. Password fields (AXSecureTextField) are refused, with no override.

• Read-only apps. System Settings, Passwords and Keychain Access can be read but never acted in, with no override. The Claude desktop app needs allow_other_session=True, because acting there uses another Claude session's permissions.

• Ambiguous matches are listed, never guessed. Pass index only after reading the list.

What we learned building it

Most of this isn't written down anywhere we could find.

• The permission goes to /usr/libexec/sshd-keygen-wrapper. Not to Terminal, not to Python, not to sshd.

• SSH runs in a background session. From there, asking the system-wide Accessibility element "which app is focused?" fails with kAXErrorCannotComplete. Asking each app directly (AXUIElementCreateApplication(pid)) works, with pids from lsappinfo, which needs no permission.

• A locked screen doesn't fail, it lies. Reads succeed, but every window title comes back as its app's name. Every reply here ends with a LOCKED note when CGSSessionScreenIsLocked is set.

• Decode AXUIElement references. They have their own CoreFoundation type. A decoder that only knows strings, numbers, booleans and point/size values silently returns nothing for AXMenuBar, AXParent and AXFocusedWindow.

• Invisible characters break matching. Calculator's display reads U+200E followed by "0". It looks like "0" and never matches it. Names are normalised (format characters dropped, whitespace collapsed) before matching.

• Xcode 26's SwiftUI panes are readable this way. AppleScript's System Events saw no text or buttons in the Organizer's archive list. This tool read all of it: rows, versions, statuses, Distribute App, Validate App. The content sits inside SwiftUI hosting views, which is our guess at why, not confirmed.

• Windows' built-in ssh-keyscan can't talk to current macOS. Windows OpenSSH 9.5 fails to negotiate a key exchange with macOS's OpenSSH 10.3 (unsupported KEX method sntrup761x25519-sha512@openssh.com), and Git for Windows' ssh-keyscan sometimes hung past its own timeout. Plain ssh works with both, so identity is checked with ssh itself.

• WebView apps expose nothing in the Simulator. A Capacitor app's content group had zero children while the screenshot showed a full page. simulator_screenshot exists so a read can be checked against something that doesn't come from Accessibility.

Tested

With macOS 26.6 (OpenSSH 10.3) as the Mac, driven from Windows 11.

• Identity: the real Mac is recognised through both Windows OpenSSH's ssh and Git for Windows' ssh. A freshly generated decoy key pinned in its place is rejected through both, and ssh -v shows it stopping at the host-key check with no key offered for login. An unroutable address is reported as a connection failure, not a mismatch.

• Presses: on Calculator, 7 was pressed once and confirmed by a separate read. All Clear pressed at 0 came back unconfirmed, correctly, so the confirmation can fail. With the press replaced by a recorder: a timeout sends AXPress only, "unsupported" sends AXPress then AXPick, and the old "any error falls through" rule is caught sending all three.

• Writes: a value set in a TextEdit file read back identically. Password fields: refused, zero writes.

• Refusals: System Settings and the Claude app refused as described.

Linux and macOS as the driving computer should work (it only needs ssh and Python), but haven't been tried.

Not yet tested

• Presses inside a native iOS app in the Simulator. Its elements offer AXPress; none has been sent yet.

• Menus and sheets that open in a separate window after a press. The confirmation only watches the pressed window, so it may say "unconfirmed" for a press that worked. Read the screen before doing anything else.

• Xcode Cloud prompts. "Xcode Cloud can automate app signing and distribution" appears on Product → Archive in Xcode 26. Whether its buttons are readable here isn't known yet. Its default button is "Get Started…", so never press Enter at it blind.

Related

uia-reader does the same for native Windows windows, through UI Automation.

License

MIT