AppKit Inspector

AppKit Inspector is a Debug-only bridge for inspecting a live native macOS AppKit interface from Codex. It captures the application's complete window frame or content view in-process, maps a clicked point to its NSView, shows hierarchy and geometry, and prepares precise visual feedback for a coding task.

Public Preview: APIs, plugin packaging, and presentation behavior may change before 1.0. Codex Browser is the supported surface. Fullscreen remains experimental and disabled by default; the plugin does not expose a system-browser/Chrome launch path.

Features

• Inspect a real AppKit window frame, title bar, traffic-light controls, toolbar, and content without Accessibility or Screen Recording permission.

• Capture real pixels by default on macOS 14.4 or later from the inspected Debug process's own WindowServer window through ScreenCaptureKit, without Screen Recording permission or access to other apps. Content is cropped from that same exact image, so appearance stays consistent.

• Choose Active Appearance to temporarily make the inspected window key for each capture, record its real emphasized AppKit state, and cooperatively return focus to the previous application. This is explicit because macOS cannot keep two applications active simultaneously.

• Target and window monitoring always uses the current inactive appearance; returning to an Inspector tab never activates the inspected application in the background.

• If exact capture is unavailable, automatically keep the Inspector usable with an honestly labeled compatibility preview and the failure reason. The legacy Hybrid/View Cache paths are fallback implementation details rather than normal UI choices.

• Click the captured interface and identify the deepest native NSView.

• Follow transient AppKit windows such as NSPopover, sheets, and panels automatically. The Inspector captures a newly appeared transient window while it is live, preserves the last snapshot after it closes, and offers a window picker for switching back to the main window.

• Select and comment individual NSTableHeaderCell regions even though AppKit draws column headers as cells rather than independent views.

• Review class names, frames, accessibility metadata, and ancestor paths.

• Use Codex Browser's native comments to annotate one or many semantic AppKit view targets and deliver them directly to the current Codex task. AppKit Inspector does not duplicate that UI with its own notes, queue, clipboard, or send controls.

• Keep discovery and transport authenticated on 127.0.0.1.

• Compile the probe out of active Release behavior with #if DEBUG.

Related MCP server: computer-use

Requirements

• macOS 14 or later;

• Xcode 26 or later with Swift 6.2;

• Node.js 22 or later;

• a current Codex desktop installation for the plugin workflow.

Quick start

Clone and verify the project:

git clone https://github.com/JaminZhou/AppKitInspector.git
cd AppKitInspector
npm ci
npm run check

Install the cloned repository as a local Codex marketplace and enable the plugin:

codex plugin marketplace add "$PWD"
codex plugin add appkit-inspector@appkit-inspector-dev

Start the included AppKit demo:

make demo

Start a new Codex task so it loads the newly installed MCP tools, then ask:

Use AppKit Inspector to connect to the running demo and open it in Codex Browser. The workflow
explicitly presents the right Browser panel after navigation. In the Inspector, use **Fit**, **−**,
and **+** to resize the snapshot; trackpad pinch and Command-modified scrolling also zoom.
Use **Window** for title-bar and frame inspection or **Content** for a focused content-only view.
Both scopes use the current process's real WindowServer pixels by default; Content crops the same
capture to the application content view. If the OS cannot provide exact pixels, the Inspector shows
an honest compatibility-preview badge and reason. Enable **Active Appearance** when selection,
focus, or toolbar emphasis must match the inspected app's foreground appearance.
Leave the window picker on **Automatic** to follow a newly opened popover, sheet, or panel, or
choose a specific visible window to keep inspection pinned there.
In Codex Browser, use its native comment mode. AppKit Inspector exposes each reviewable native view
as a semantic comment target, so comments reach the current task with the specific AppKit class and
hierarchy instead of targeting the whole screenshot. Multiple native comments can be submitted as
one Codex message. Ordinary clicks still select views for hierarchy and geometry inspection.

The default open_appkit_inspector tool creates a 60-second, single-use loopback URL for the current task's right Browser panel. It never requests fullscreen or invokes the system browser.

Codex integration

AppKit Inspector is currently adapted specifically for Codex desktop:

• open_appkit_inspector prepares the authenticated URL that Codex opens in its right Browser panel;

• transparent semantic targets map Codex Browser comments to AppKit class, hierarchy, and frame context;

• Codex owns comment composition, multi-comment submission, delivery, and task history;

• the Inspector owns capture, view selection, hierarchy, geometry, and target refresh only.

This boundary deliberately avoids maintaining a second annotation system with different delivery and recovery behavior. If a host does not provide Codex Browser native comments, inspection remains readable and selectable, but comment delivery is unavailable rather than silently falling back to the clipboard or an external browser.

Add the probe to an AppKit project

Add the package in Xcode with:

https://github.com/JaminZhou/AppKitInspector.git

Or declare it in Package.swift:

.package(
    url: "https://github.com/JaminZhou/AppKitInspector.git",
    from: "0.1.1"
)

Add AppKitInspectorProbe only to a Debug target or Debug configuration, then start it after the application has launched:

#if DEBUG
import AppKitInspectorProbe

_ = try? AppKitInspectorProbe.start()
#endif

Never add or start the probe in Release, archive, TestFlight, or App Store builds.

Presentation modes

• Codex Browser: supported product path.

• MCP App fullscreen: experimental, disabled by default, and available only when the MCP server starts with APPKIT_INSPECTOR_EXPERIMENTAL_FULLSCREEN=1.

No presentation failure opens Chrome, Safari, or another external window.

Repository layout

• native/ — reusable Swift probe, demo application, and native tests.

• packages/mcp/ — local MCP server, discovery, and authenticated transport.

• packages/app/ — Inspector browser interface and local client.

• plugins/appkit-inspector/ — installable Codex plugin, generated distribution, and skill.

• TODO.md — deferred work and experimental fullscreen acceptance.

Development

npm ci
npm run check
swift build -c release

For a live bridge check, run make demo in one terminal and npm run test:live in another. After changing packages/app/ or packages/mcp/, run npm run build and include the matching generated files under plugins/appkit-inspector/dist/. The authenticated Browser uses a lightweight target status request to notice when the inspected Debug application is rebuilt; it follows a unique replacement with the same bundle identifier and refreshes the snapshot without reloading the page.

Security and privacy

The probe and Inspector servers bind only to loopback, require random credentials, and write user-private discovery data. Codex Browser launch links are single-use and become HttpOnly, same-site sessions. The project uses public Apple SDK APIs and does not use injection, Accessibility automation, private frameworks, or Screen Recording permission. Exact capture is restricted to SCShareableContent.currentProcess, so it cannot enumerate or capture another process's windows.

Captured screenshots, hierarchy data, and Codex Browser comments are sensitive. When used through Codex they may become part of the Codex task under the user's product and workspace data controls. Read SECURITY.md before integrating the probe and use private security advisories for vulnerability reports.

Contributing

See CONTRIBUTING.md. Contributions intentionally submitted to the project are licensed under Apache License 2.0.

License

Copyright 2026 Jamin Zhou.

Licensed under the Apache License, Version 2.0. Bundled dependency attributions are documented in THIRD_PARTY_NOTICES.md.