Peekaboo MCP

--- summary: 'Review Remote Testing Playbook guidance' read_when: - 'planning work related to remote testing playbook' - 'debugging or extending features described here' --- ## Remote Testing Playbook This document captures the current workflow for running Peekaboo’s SwiftPM test targets on a remote macOS VM over SSH, plus the pitfalls we hit while bringing the VM online. ### 1. Prerequisites - **Network access**: the VM must be reachable via Tailscale. Verify with `ping 100.64.183.103` (replace with your tailnet IP). - **SSH key**: copy your workstation key to the VM (`~/.ssh/authorized_keys`, 600 permissions). All commands below assume you can run `ssh steipete@peters-virtual-machine`. - **Toolchains**: the VM needs Xcode/Swift toolchains that bundle the Swift Testing framework. On the VirtualBuddy instance we set the command line tools explicitly: ```bash sudo xcode-select --switch /Applications/Xcode.app xcode-select -p # confirm path is /Applications/Xcode.app/Contents/Developer ``` - **Homebrew (optional)**: installed via `curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh | /bin/bash` so we can add tooling later (tmux, pnpm, etc.). - **Privacy permissions**: macOS only surfaces Accessibility / Screen Recording prompts in the GUI session. If you skip this step when driving tests over SSH, the CLI is denied access and the suite hangs. See [Granting privacy permissions](#granting-privacy-permissions-required-for-automation). ### 2. Sync the Repository From the local checkout: ```bash rsync -az --delete \ --exclude '.build' --exclude 'DerivedData' --exclude '.DS_Store' \ ./ steipete@peters-virtual-machine:Projects/peekaboo ``` This keeps the remote tree in lock-step with `main`, including submodules. ### 3. Running the “Safe” (Non-Automation) Test Set ```bash ssh steipete@peters-virtual-machine \ 'cd ~/Projects/peekaboo/Apps/CLI && swift test -Xswiftc -DPEEKABOO_SKIP_AUTOMATION' ``` Hints: - The `-DPEEKABOO_SKIP_AUTOMATION` flag matches local CI defaults and compiles only `CoreCLITests`. - With Swift 6.2 / Swift Testing we had to enable the feature in `Package.swift` via `.enableExperimentalFeature("SwiftTesting")`. Without that, the remote build dies with `no such module 'Testing'`. - If you want a log, send output to a file (`… > /tmp/peekaboo-safe.log`). ### 4. Running read-only automation checks To exercise commands that only query system state (help text, listing apps/spaces, JSON output validation) without triggering UI changes, run: ```bash pnpm run test:automation:read ``` This sets `RUN_AUTOMATION_READ=true` and executes the automation target. Tests that would click, drag, or launch apps are skipped. ### 5. Running the Full Automation Suite If you just want the CLI automation target (without local UI interaction), the existing script still works: ```bash ssh steipete@peters-virtual-machine \ 'cd ~/Projects/peekaboo/Apps/CLI && PEEKABOO_INCLUDE_AUTOMATION_TESTS=true swift test' ``` For **full local automation** (UI-driven cases that expect a real display) we added a convenience script to `package.json`. It builds the CLI, points tests at the actual binary, and sets the right env vars: ```bash pnpm run test:automation:local ``` This must be executed either: - From an interactive Terminal on the VM (preferred), or - Over SSH after privacy permissions have been granted and you’ve started a tmux session to keep the job alive. `pnpm run test:automation:local` writes logs to `~/Projects/peekaboo/logs/automation-<timestamp>.log`. Tail the file while it runs to watch progress. Warnings & learnings: - The automation suite is heavy and may hang the VirtualBuddy UI if permissions are missing; watch the log for stalled commands. - Running inside tmux (`brew install tmux`) is recommended so a frozen SSH session doesn’t kill the run. - Grant Accessibility/Screen Recording before launching the script; otherwise macOS silently denies UI automation. ### 6. Diagnosing the Remote Environment - `xcode-select -p` confirms which command line tools SwiftPM uses. - `swift --version` prints the Swift toolchain (currently Swift 6.2.1 on the VM). - If you need a visual check, Peekaboo can ironically be pointed at the VirtualBuddy UI to screenshot status dialogs. ### 7. Known Issues & Follow-up - **Automation freeze**: investigate why `swift test` stalls during automation runs in VirtualBuddy (possibly accessibility permissions or long-running UI automation). - **Tooling gaps**: install tmux, pnpm, and poltergeist services on the VM for parity with the Mac Studio workflow. - **Logs**: standardize capturing test output under `/tmp/peekaboo-*.log` so multiple operators can review results. - **Risky suites**: see the table below—anything marked *High* should only run on a disposable VM snapshot. ### Quick Checklist 1. `ssh steipete@peters-virtual-machine` works (authorized key + Tailscale). 2. `xcode-select -p` → `/Applications/Xcode.app/Contents/Developer`. 3. `rsync … ./ steipete@peters-virtual-machine:Projects/peekaboo`. 4. Safe suite: `swift test -Xswiftc -DPEEKABOO_SKIP_AUTOMATION`. 5. Automation suite (optional): `PEEKABOO_INCLUDE_AUTOMATION_TESTS=true swift test` (watch for hangs). 6. Capture output for each run and file it in `/tmp` for later inspection. Following this flow we successfully ran the non-automation tests remotely; automation still needs stabilization once the VM finishes freezing issues. ### Granting privacy permissions (required for automation) macOS’ Transparency, Consent, and Control (TCC) framework **never** displays prompts to headless sessions. Launching automation over SSH without first approving the binaries leads to immediate hangs because helper processes (e.g. `swift-run peekaboo …`) are denied Accessibility / Screen Recording access. Fix: 1. Connect via Screen Sharing or VirtualBuddy and log in as the test user. 2. Open **System Settings → Privacy & Security** and visit **Accessibility**, **Screen Recording**, and **Automation** (optionally **Full Disk Access** if needed). 3. Add and enable these executables (update paths if SwiftPM rebuilds into a new folder): ``` ~/Projects/peekaboo/Apps/CLI/.build/arm64-apple-macosx/debug/peekaboo ~/Projects/peekaboo/Apps/CLI/.build/arm64-apple-macosx/debug/peekabooPackageTests.xctest/Contents/MacOS/peekabooPackageTests /Applications/Xcode.app/Contents/Developer/usr/bin/swift /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift ``` 4. Relaunch the automation suite (from SSH or local Terminal); no prompts reappear once these executables are approved. > There is no supported way to approve these prompts purely over SSH. If GUI access is impossible, pre-approve via an MDM/PPPC profile or script the System Settings UI while logged in locally. ### Automation suite risk map | Suite (file) | Env gate | UI/system impact | Risk | |--------------|----------|------------------|------| | `AgentIntegrationTests.swift` | `RUN_AGENT_TESTS=true` + LLM API key | Launches TextEdit/Safari, types, minimizes windows | **High** | | `AgentMenuTests.swift` | `RUN_LOCAL_TESTS=true` + LLM API key | Launches Calculator/TextEdit, agent drives menus | **High** | | `AppCommandTests.swift` (integration section) | `RUN_LOCAL_TESTS=true` | Launch/quit/hide/show TextEdit (with `--save-changes`) | **High** | | `DragCommandTests.swift` (integration section) | `RUN_LOCAL_TESTS=true` | Real drag gestures, can drop to Trash | **High** | | `FocusIntegrationTests.swift`, `ClickCommandFocusTests.swift` | `RUN_LOCAL_TESTS=true` | Generates mouse/keyboard events to focus Finder/TextEdit | **High** | | `MenuCommandTests.swift` (integration) | `RUN_LOCAL_TESTS=true` | Navigates Finder menus | **Medium–High** | | `DialogCommandTests.swift` (integration) | `RUN_LOCAL_TESTS=true` | Interacts with active dialogs | **Medium** | | `WindowCommandTests.swift` (local integration) | `RUN_LOCAL_TESTS=true` | Moves/minimizes TextEdit windows | **Medium** | | `SeeCommandAnnotationIntegrationTests.swift` | Disabled by default; needs `RUN_LOCAL_TESTS=true` | Launches Safari to capture screenshots | **Medium** | | `ScreenshotValidationTests.swift`, `AnnotationIntegrationTests.swift` | `RUN_LOCAL_TESTS=true` | Creates temporary NSWindows | **Low** | | CLI parsing/JSON/configuration suites | none | Pure logic | **Low** | Only enable the *High*-risk suites when you’re inside a dedicated VM snapshot and expect the UI to shift. Leave `RUN_AGENT_TESTS` unset unless you specifically want to exercise agent-driven flows.