Peekaboo MCP

--- summary: 'Runtime logger + Visualizer refactor log' read_when: - Coordinating CLI runtime injection - Tracking Visualizer client fixes --- ## Progress (Nov 2025) - **Nov 10 2025 (build `cli-build-1762780242`):** SpaceCommand now matches the CLI runtime pattern (structs hold state, `@MainActor run(using:)`, conformances in nonisolated extensions). Current blockers are the menu/system shells: `MenuCommand` subcommands still declare `@MainActor extension … : AsyncRuntimeCommand, OutputFormattable`, causing both `#ConformanceIsolation` and redundant conformances. Next steps: (1) reapply the struct+extension pattern to every `MenuCommand` subcommand, replacing the `@MainActor` conformances with plain extensions; (2) repeat for Dock/MenuBar/Run/Sleep once menu is clean; (3) only after those files compile should we revisit WindowCommand to make sure each subcommand follows the same template. - **Nov 10 2025 (pending build)**: MenuCommand and all four subcommands now follow the runtime template (plain structs with `@RuntimeStorage`, `@MainActor run(using:)`, and nonisolated configuration builders via `MainActorCommandDescription`). Menu interactions route through `MenuServiceBridge`, eliminating direct singleton access. Next up: apply the same structure to Dock/MenuBar/Run/Sleep before kicking off another `./runner tmux … scripts/tmux-build.sh` run to see how far the build gets. - **Nov 10 2025 (pending build)**: MenuCommand and all four subcommands now follow the runtime template (plain structs with `@RuntimeStorage`, `@MainActor run(using:)`, and nonisolated configuration builders via `MainActorCommandDescription`). Menu interactions route through `MenuServiceBridge`, eliminating direct singleton access. Next up: apply the same structure to Dock/MenuBar/Run/Sleep before kicking off another `./runner tmux … scripts/tmux-build.sh` run to see how far the build gets. - **Nov 10 2025 (still pending build)**: DockCommand has been rewritten to the same pattern. Each Dock subcommand now caches `CommandRuntime`, executes on the main actor, and calls `DockServiceBridge` (no `PeekabooServices()` reads). Remaining system shells to convert before the next build: MenuBarCommand, RunCommand, and SleepCommand. - **Nov 10 2025 (pending build)**: MenuBarCommand no longer relies on `@MainActor MainActorAsyncParsableCommand`; it is now a plain `ParsableCommand` with a runtime-backed `run(using:)` and calls into `MenuServiceBridge` for listing/clicking status items. Remaining shells to migrate before the next build: RunCommand and SleepCommand. - **Nov 10 2025 (build `cli-build-1762781419`)**: RunCommand and SleepCommand now follow the runtime pattern (plain structs, `@RuntimeStorage`, method-level `@MainActor`). After rerunning `./runner tmux new-session -d -s cli-build-1762781419 ./scripts/tmux-build.sh`, the build progressed to the expected `WindowCommand` conformances: `Move`, `Resize`, `SetBounds`, and `WindowList` still declare `struct FooSubcommand: AsyncParsableCommand, AsyncRuntimeCommand ...` with type-level isolation, so Swift 6.2 emits `#ConformanceIsolation`. Next focus: convert those WindowCommand subcommands (and any siblings still using inline conformances) to the struct+extension template so we can finally reach the Visualizer crash. - **Nov 10 2025 (build `cli-build-1762782321`)**: WindowCommand subcommands (close/minimize/maximize/focus/move/resize/set-bounds/list) are now nested under `extension WindowCommand`, store `CommandRuntime` via `@RuntimeStorage`, and declare `run(using:)` as `@MainActor`. The tmux build advanced further but now fails because (a) the repo has two `WindowServiceBridge` definitions (the legacy one in `WindowCommand.swift` conflicts with the shared version in `CommandUtilities.swift`), and (b) `SpaceCommand`’s nested structs still expose their conformances inside the enclosing extension, so Swift treats the `extension SpaceCommand.*` blocks as invalid and the `subcommands:` array as `[Any]`. Next tasks: drop the duplicate `WindowServiceBridge` (reuse the bridge in `CommandUtilities`) and lift the SpaceCommand conformances to file scope (or wrap the structs in their own `extension` blocks like we did for WindowCommand) before rerunning the tmux build. - **Nov 10 2025 (build `cli-build-1762783615`)**: SpaceCommand, RunCommand, SleepCommand, and AgentCommand now follow the runtime pattern (plain structs, `@RuntimeStorage`, `@MainActor run(using:)`, conformances declared via `extension Foo: @MainActor AsyncParsableCommand/AsyncRuntimeCommand`). AgentOutputDelegate shed the blanket `@MainActor` so it can override PeekabooCore formatters without isolation errors, and SeeCommand’s runtime conformance is now a single `extension SeeCommand: @MainActor AsyncRuntimeCommand`. The latest tmux run progresses past Space/Window/Agent to the MCP command set; every MCP subcommand still conforms inline (`struct MCPCommand.Remove: AsyncRuntimeCommand`) so Swift 6.2 throws `#ConformanceIsolation`. Next up: rewrite each MCP subcommand to the same struct+extension template, then rerun the build to (hopefully) hit the Visualizer crash. - **Nov 10 2025 (build `cli-build-1762784968`)**: MCPCommand.Remove/Test/Info/Enable/Disable/Inspect, Dock/MenuBar/Dialog subcommands, Run/Sleep/Agent/Space/Window/List all now use the runtime template (`@RuntimeStorage`, `@MainActor run(using:)`, `extension Foo: @MainActor AsyncParsableCommand/AsyncRuntimeCommand`). The build moves on to the next batch of older commands still using inline conformances: ToolsCommand, ClickCommand, and the remaining Menu/Dialog helpers need the same treatment. AgentOutputDelegate’s `UnknownToolFormatter` overrides now declare `nonisolated override` but still need the flag on `formatStarting`/`formatCompleted`. Next step: convert the remaining CLI roots (ToolsCommand, ClickCommand, MenuCommand.Click/ClickExtra, DialogClick etc.) to the shared runtime structure, add `@MainActor` to their conformances, then rerun the tmux build to verify we finally hit the Visualizer crash. - **Nov 10 2025 (build `cli-build-1762785297`)**: ToolsCommand, ClickCommand, MenuCommand.Click/ClickExtra/List/ListAll, and all Dialog subcommands now match the runtime template; `UnknownToolFormatter` overrides are fully `nonisolated`. The build gets past menu/dialog/interaction code and now fails on `ConfigCommand` (Init/Show/Edit/Validate/SetCredential/AddProvider/ListProviders/TestProvider/Models) because their conformances are still inline `struct Foo: AsyncRuntimeCommand`. Next focus: refactor the remaining ConfigCommand subcommands to the new pattern so we can progress toward the Visualizer crash. - **Nov 10 2025 (build `cli-build-1762786168`)**: Interaction commands (drag/hotkey/move/press/scroll/swipe/type) still used type-level `@MainActor` conformances, triggering `#ConformanceIsolation`, and DragCommand duplicated its `OutputFormattable` conformance. Action: convert every interaction command to the runtime template (plain structs + `@RuntimeStorage`, method-level `@MainActor run(using:)`, conformances declared via `extension Foo: @MainActor AsyncParsableCommand/AsyncRuntimeCommand`) so they stop depending on singleton loggers/services. - **Nov 10 2025 (build `cli-build-1762786810`)**: After converting the interaction commands, the build advanced to `PermissionCommand`. Subcommands were still top-level structs, so the `extension PermissionCommand.*` conformances referenced non-existent nested types. Action: nest the status/request subcommands inside `PermissionCommand`, keep them on the runtime template, then rerun the build. - **Nov 10 2025 (build `cli-build-1762786927`)**: CLI now stalls in `ListCommand` + `ImageCommand`. The remaining list subcommands (permissions/menubar/screens) still used inline conformances and singleton loggers, and `ImageCommand` both conformed directly to `AsyncRuntimeCommand` and re-declared the conformance in an extension. Fix plan: convert every `ListCommand` subcommand to the runtime template (with `MainActorCommandDescription` builders), reimplement permission listing via `PermissionHelpers`, rework menu bar output to avoid optional-title warnings, and migrate `ImageCommand` to the same pattern (including a proper `ensureFocused` options argument). Once those compile, rerun the tmux build to discover the next blocker. - **Nov 10 2025 (build `cli-build-1762788113`)**: Finished the CLI sweep—converted ListCommand (apps/windows/permissions/menubar/screens), MCP list/add, PermissionsCommand, LearnCommand, ImageCommand, and the interaction/focus helpers to the runtime template, rewired permission helpers to `PermissionHelpers`, and fixed the MCP metadata fields. The tmux build now completes (just logs the duplicate swift-argument-parser warning). Next step: tackle the swift-argument-parser duplication by pointing PeekabooCore (and downstream packages) at the vendored fork so we don’t drift back to Apple’s upstream when building CLI. - **Nov 10 2025 (build `cli-build-1762788362`)**: Resolved the duplicate swift-argument-parser identity warning by updating every package manifest that previously referenced the GitHub fork (`AXorcist`, `Core/PeekabooExternalDependencies`, `Examples/Package.swift`) to depend on the vendored checkout (`Vendor/swift-argument-parser`). Re-ran the tmux build and confirmed it now finishes cleanly with zero warnings. Next dependency task: audit any other repos (e.g., future example targets) if new manifests appear, but for now the CLI build graph is fully on the vendored parser so we can safely continue toward the Visualizer work. - **Nov 10 2025 (build `cli-build-1762788843`)**: (Historic) VisualizationClient once again talks to the LaunchAgent broker (`PeekabooBridgeHost`) instead of attempting to connect directly to the app’s anonymous listener. The client now fetches `NSXPCListenerEndpoint`s from the broker, validates them, and only then spins up the direct connection, so CLI builds (and runtime visual feedback) no longer hang when the anonymous listener moves. Follow-up: silence the remaining Swift 6 warnings (`any Encoder` in `CommandRuntime`, `await` with no suspension) when we tighten the language mode. - **Nov 10 2025 (build `cli-build-1762789082`)**: Cleaned up the Swift 6 warning backlog—`RuntimeStorage` now uses `any Encoder/Decoder`, VisualizationClient’s broker helpers use `any VisualizerEndpointBrokerProtocol`, and SpaceCommand’s helper actor hops go through `MainActor.run` (with a real await for `switchToSpace`). AgentCommand’s redundant `await`/`try` sites were simplified. The tmux build is back to warning-free aside from deliberate TODOs (SpaceUtilities, AgentCommand telemetry). Next step: if we want zero warnings, migrate SpaceUtilities’ `Task { @MainActor [buffer]` block to the new `withTaskCancellationHandler` pattern, but it’s not blocking the CLI. - **Nov 10 2025 (build `cli-build-1762789445`)**: ScreenCaptureService’s `CaptureOutput` no longer captures a non-Sendable `CMSampleBuffer` inside a detached `Task { @MainActor … }`. We now extract the pixel buffer on the capture thread, build the `CGImage`, and only hop to the main actor when resuming the continuation. Result: the last Swift 6 warning is gone and the capture timeout logic is still intact. - **Nov 10 2025 (build `cli-build-1762790361`)**: Default visualizer animation speed bumped from 1.0× to **1.4×** (via `PeekabooSettings.defaultVisualizerAnimationSpeed`). VisualizerCoordinator now uses that constant everywhere it previously hard-coded `?? 1.0`, so fresh installs linger a bit longer and docs reflect the slower pacing. - **Nov 10 2025 (build `cli-build-1762791510`)**: Per feedback, the slider defaults to 1.0× again, but VisualizerCoordinator now applies an internal 1.4× multiplier so “1×” still looks like the slower pacing. This keeps the UI intuitive while preserving the more visible animations we just shipped. - **Nov 10 2025 (build `cli-build-1762794091`)**: Rebalanced the visualizer so the slider’s **1.0×** default actually looks good. Each animation now has a human-friendly baseline (flash ≈0.35 s, click ripple ≈0.45 s, swipe/mouse trail ≈0.9 s, etc.) and the slider simply scales those baselines. Docs highlight the baselines instead of hiding multipliers. - **Nov 11 2025 (build `cli-build-1762795001`)**: Dropped the LaunchAgent + AsyncXPCConnection bridge. `VisualizationClient` now serializes `VisualizerEvent` payloads, writes them to `~/Library/Application Support/PeekabooShared/VisualizerEvents`, and pings Peekaboo.app via `NSDistributedNotificationCenter`. The app listens through `VisualizerEventReceiver`, loads the JSON, relays to `VisualizerCoordinator`, then deletes the file. Added storage overrides (`PEEKABOO_VISUALIZER_STORAGE`, `PEEKABOO_VISUALIZER_APP_GROUP`) and background cleanup so abandoned events vanish automatically. ## Archived refactor highlights (Nov 2025) - **AgentCommand split (Nov 17)** — extracted chat/audio flows, added launch policy scaffolding; needs tighter cancellation and more UI/tests (see `docs/archive/refactor/agent-command-split.md`). - **ConfigCommand split/refactor (Nov 17)** — broke the 1.2k-line command into subcommands and helpers; next steps include shared MCP client service and consistent formatting/error handling (`config-command-split.md`, `config-refactor-2025-11-17.md`). - **MCPCommand split (Nov 17)** — per-subcommand files plus shared parsing/formatting helpers; add mockable MCPClientService and broaden tests. - **MenuService refactor (Nov 18)** — split traversal/actions/extras, added traversal budgets; needs ContinuousClock timings, stronger title matching, and injected visualizer/logger seams. - **AXorcist boundary logs (Nov 19 + undated)** — keep AXorcist lean (AX toolkit) and push heuristics into Peekaboo; catalog follow-ups in `axorcist-2025-11-19.md` and `axorcist.md`. - **Agent improvements (pi-mono learnings, Nov 21)** — queue mode, streamed tool-call diffs/redaction, event unification, and session logging plans (`agent-improvements.md`). - **Capture follow-ups** — watch → capture migration follow-ups and test gaps (`capture-todo.md`). - **Open/app launch tests** — WIP abstraction/test plan for `open` vs `app launch` flows (`open-launch-tests.md`). - **Tool results refactor** — richer ToolResponse formatting for agent outputs (`tool-results.md`).