mac-pilot-mcp

Read / write / clear the macOS clipboard (text only, via pbpaste/pbcopy).

Examples:

• Read text: { action: "read" }

• Write text: { action: "write", text: "hello" }

• Clear: { action: "clear" }

Limitations:

• Text only. Images/files in the clipboard appear as their typed name.

• No clipboard history — only the current value.

Find UI elements (role, title, position, size) in an app via Accessibility, with optional Electron CDP fallback.

Examples:

• All buttons in Safari: { app: "Safari", role: "AXButton" }

• Element by title: { app: "Mail", title: "Send" }

• Fuzzy search: { app: "Finder", searchText: "Documents" }

• Electron AX (VSCode with --remote-debugging-port=9222): { app: "Visual Studio Code", useElectronFallback: true }

• Auto-fallback for known Electron apps: { app: "Cursor", useElectronFallback: "auto" }

Limitations:

• Requires Accessibility permission for the client app.

• Electron apps (VSCode, Cursor, Slack, Discord) expose a thin AX tree; use useElectronFallback when AX returns empty.

• CDP fallback needs the user to relaunch the app with --remote-debugging-port=.

Check macOS Privacy permissions (Automation/Accessibility/Screen Recording) + deep-link to grant.

Examples:

• { check: "all" } → return state of all 3 permissions

• { check: "accessibility" } → only Accessibility

Limitations:

• macOS doesn't expose TCC state to ordinary processes. We probe by attempting a known-safe operation and observing the error class.

• "Screen Recording" cannot be detected reliably without invoking a screenshot; we report "unknown" instead of guessing.

Export user-saved recipes as a portable .mac-recipe.json bundle (built-ins excluded by default).

Examples:

• Export one recipe inline: { name: "open-url-in-private" }

• Export all user recipes to a file: { outputPath: "~/recipes/backup.json" }

• Include built-ins: { outputPath: "~/recipes/full.json", includeBuiltins: true }

Format: mac-recipe-bundle/v1 (versioned for forward compatibility). Limitations: outputPath must resolve under $HOME and end with .json.

Import a .mac-recipe.json bundle (inline or file path). Conflict policy: skip | rename | replace.

Examples:

• From file: { inputPath: "~/recipes/team-bundle.json" }

• From file with replace policy: { inputPath: "~/recipes/team.json", onConflict: "replace" }

• Inline bundle: { bundle: { format: "mac-recipe-bundle/v1", exportedAt: "...", recipes: [...] } }

Conflict policy:

• "skip" (default): keep existing recipe

• "rename": append "-imported-N" until name is unique

• "replace": delete existing first

Limitations:

• Provide exactly one of bundle or inputPath.

• Bundle format major version is enforced — v2 bundles rejected.

Run a saved recipe by name with parameter values.

Examples:

• { name: "toggle-dark-mode" }

• { name: "open-url-in-private", params: { url: "https://example.com" } }

• Dry run (preview steps): { name: "send-email", params: { ... }, dryRun: true }

Limitations:

• Stops on the first failed step (no partial rollback).

• Each step inherits the recipe's app as appContext for self-learning.

Save a multi-step automation as a named recipe. {{param}} placeholders are JSON-safe substituted.

Example: { name: "open-url-in-private", description: "Open a URL in Safari private window", app: "Safari", steps: [ { actionType: "open", params: { target: "Safari" }, description: "Launch Safari" }, { actionType: "keypress", params: { text: "cmd+shift+n" }, description: "Open private window" }, { actionType: "keypress", params: { text: "cmd+l" }, description: "Focus address bar" }, { actionType: "type", params: { text: "{{url}}" }, description: "Enter URL" }, { actionType: "keypress", params: { text: "return" }, description: "Navigate" } ], parameters: [{ name: "url", description: "URL to open" }], tags: ["safari", "browser", "private"] }

Limitations:

• Recipe names are unique. Re-saving the same name fails — delete or rename.

• Each step's params goes through the security sandbox at run time.

Search 118 built-in + saved recipes by natural-language query. Call this BEFORE writing new AppleScript.

Examples:

• { query: "dark mode" } → matches toggle-dark-mode, get-dark-mode

• { query: "screenshot", app: "Finder" }

• { query: "send email", includeHistory: true }

Limitations:

• FTS5 tokenizer is English-biased. For Korean/CJK, supplement with the app filter.

Run an AppleScript/JXA/shell command or send click/type/keypress to macOS.

Supported actionTypes: applescript, jxa, shell, open, click, type, keypress.

Examples:

• Open Safari: { actionType: "open", target: "Safari" }

• Type text: { actionType: "type", text: "hello world" }

• Cmd+C: { actionType: "keypress", text: "cmd+c" }

• Run AppleScript: { actionType: "applescript", script: 'tell application "Finder" to activate', appContext: "Finder" }

• List files: { actionType: "shell", command: "ls -la ~/Documents" }

Limitations:

• Requires Accessibility + Automation permissions for click/type/keypress and AS targeting other apps.

• Cannot bypass the lock screen.

• Dangerous shell patterns (rm -rf /, curl|sh, sudo, etc.) are blocked by the sandbox.

• Set appContext for the best self-learning: errors get classified per-app and reliable hints are auto-prepended on subsequent calls.

Capture screen, window, or region as base64 PNG.

Examples:

• Full screen: { target: "screen" }

• Specific app window: { target: "window", windowName: "Safari" }

• Region (x,y from top-left): { target: "region", region: { x: 0, y: 0, width: 800, height: 600 } }

• High-fidelity capture: { target: "screen", scale: 1.0 } (warning: larger token cost)

Limitations:

• Requires Screen Recording permission (System Settings → Privacy & Security → Screen Recording).

• Default scale is 0.5 to keep token cost reasonable; bump for OCR-quality captures.

• Cursor is not included in the capture.

Read current macOS state: frontmost app, windows, Finder selection, running apps.

Examples:

• All state: {} (returns everything)

• Window list only: { include: ["windows"] }

• Frontmost app: { include: ["frontmost_app"] }

Limitations:

• Window list requires Accessibility for the apps you want to see.

• "clipboard" still works but mac_clipboard is preferred (lighter + write support).