MarkuprPlus

The Problem

Your coding agent can't see your screen. So you stop working and start transcribing: describe the layout bug in prose, take a screenshot, crop it, drag it into the right place, explain which part matters. You speak at 150 words per minute and type at 60, and the context leaks out on the way.

Related MCP server: autodemo

The Solution

MarkuprPlus is a menu bar app. It records the exact window you point at, listens while you narrate, and lets you draw on the live screen without blocking your clicks. When you stop, it transcribes on-device, aligns your words to each mark, and writes a report your agent can act on — one finding per circle, each with its own annotated frame.

Cmd+Shift+F  →  talk  →  Cmd-drag  →  Cmd+Shift+F  →  paste into your agent

The Loop

1. Press Cmd+Shift+F and pick your target

The picker opens above every window. Whatever is under your cursor lights up — click it and the recorder locks onto that window alone, and stays locked through moves, resizes, and app switches. Region and full-display modes are one click away when a finding needs a wider frame.

If the window's identity or native geometry ever becomes ambiguous mid-session, capture stops rather than widening to whatever is behind it. You never ship a frame you didn't mean to share.

The whole app lives in this one portrait popover. No window to manage, no dock icon.

2. Talk while you work

Keep clicking through your app as usual. MarkuprPlus records the window and your microphone together, then transcribes with Whisper on-device after you stop.

The picker, the drawing canvas, and the recording HUD are all excluded from screen capture at the OS level — nothing MarkuprPlus draws ends up in your video. Only your app does, plus the strokes you meant to leave.

3. Hold Cmd and draw

Command-drag paints straight onto the live screen — freehand, circle, or highlight, in a colour you choose. Let go of the key and your next normal click both reaches the app underneath and saves that mark, clearing the canvas for the next one.

On Windows, hold Ctrl instead.

Three circles means three issues, not one screenshot with three scribbles on it. Each finding carries its own PNG, timestamp, tool, and colour.

4. Press Cmd+Shift+F again

Whisper runs, your narration is aligned to each mark, and the report is written to disk. The markdown path lands on your clipboard the moment it's ready — paste it straight into Claude Code, Codex, Cursor, or anything else that reads a file.

Everything for the session goes in one folder: the report, the screenshots, the session video, the narration audio, and the metadata.

Quick Start

Desktop app (recommended)

Download from markuprplus.com or the releases page. macOS on Apple Silicon and Intel, plus Windows.

macOS install note: Direct downloads from GitHub Releases are signed, notarized, and stapled. If Gatekeeper rejects an artifact, use MarkuprPlus support so the release can be investigated.

1. Press Cmd+Shift+F (macOS) or Ctrl+Shift+F (Windows) and click the window you want.

2. Narrate what you see. Hold Cmd / Ctrl and drag to mark the live screen.

3. Release the key, then click normally — that saves the mark and clears the canvas. Repeat for every finding.

4. Press the hotkey again to stop. The report path is on your clipboard.

MCP server (for AI coding agents)

npx --yes --package markuprplus markuprplus-mcp

CLI (for recordings you already have)

npx markuprplus analyze ./recording.mov

Compatibility: the public package and commands are markuprplus and markuprplus-mcp. Existing .markuprx project files, storage paths, and app identifiers remain unchanged so upgrades keep their settings and sessions.

Optional companion for Mac App Store CLI integrations

The sandboxed Mac App Store app can use AI command-line tools already installed and signed in on your Mac through the optional MarkuprPlus CLI Bridge. Local Rules, Ollama, LM Studio, and Anthropic API do not require this companion.

Install and pair it from a Terminal:

npm install -g markuprplus
markuprplus bridge install     # installs and starts a per-user LaunchAgent
markuprplus bridge token       # paste this value in Settings → Advanced
markuprplus bridge status

The public npm package contains the same versioned CLI and MCP entry points used by the source release.

The service listens only on 127.0.0.1:49647, requires its random pairing token for every provider request, and accepts a fixed structured-report protocol rather than shell text. The App Store app does not run shell commands or launch external tools; the separately installed companion invokes only the provider selected in Settings.

Lifecycle and recovery commands:

markuprplus bridge start
markuprplus bridge stop
markuprplus bridge status
markuprplus bridge token
markuprplus bridge rotate-token
markuprplus bridge uninstall

Rotating the token requires pairing again. Uninstall removes the exact LaunchAgent and bridge configuration owned by MarkuprPlus; it does not uninstall your AI CLIs.

What Lands in Your Folder

One session, one directory:

originplayer-test-iphone-20260818-104731/
├── feedback-report.md          # the file you paste
├── feedback-summary.md         # counts and duration
├── metadata.json               # per-issue capture context
├── processing-trace.json       # which provider ran, how long, why it fell back
├── screenshots/
│   ├── marked-issue-001.png    # one frame per mark, your stroke composited in
│   ├── marked-issue-002.png
│   └── marked-issue-003.png
├── session-recording.webm      # the window, with annotations
└── session-audio.webm          # your narration

A real, unedited finding from that session:

### MX-001

- **Timestamp:** 00:08
- **Tools:** freehand
- **Colors:** #ff3b30

#### User Comment

> So there's a search menu and over here by default, if there's been
> previous searches, it should list out all the searches that have
> happened in the past.

#### Marked Evidence

![Marked issue MX-001](./screenshots/marked-issue-001.png)

Each finding carries more than pixels:

Marked issues are numbered MX-001…; items that come from narration alone are FB-001….

Every Surface

Report Providers

Pick the model that turns a capture into a structured report. MarkuprPlus checks each one before you record and shows you what it actually found.

In the Mac App Store app, CLI providers use the optional local companion described above. Direct desktop builds invoke the same adapters inside the app. Both paths preserve Local Rules as the failure-safe report.

Failure is safe by design. If the provider you picked errors out, the deterministic Local rules report is written anyway, and the popover names the provider and the reason. Your recording, audio, and marks were already on disk before analysis started. An explicit CLI choice never silently becomes an Anthropic call.

Codex CLI and OpenCode can receive captured screenshots. Transcript-only CLI adapters reject screenshot-only sessions instead of inventing visual findings.

processing-trace.json records exactly what happened:

{
  "requestedProvider": "codex-cli",
  "actualProvider": "rules",
  "aiFallbackReason": "Codex analysis exited with status 1.",
  "aiEnhanced": false,
  "totalMs": 4220
}

Why MarkuprPlus

Evidence, not footage. A screen recording leaves your agent a video it can't watch and you a file you have to narrate twice. This gives you separate findings, one annotated frame each, the words you said at that timestamp, and the window and cursor context.

Local-first. Whisper runs on your device and Local rules needs no credentials. Cloud transcription and cloud models only run when you explicitly pick them. No account, no telemetry, no analytics.

It doesn't film itself. The picker, the drawing canvas, and the recording HUD are content-protected at the OS level. Your marks reach the report; the app's own chrome never reaches the video.

Works everywhere. Desktop app for daily flow, CLI for scripts and CI, MCP server for agents, GitHub Action for pull requests. One pipeline, four front doors.

Open source. MIT licensed. Read it, fork it, ship it.

MCP Server

Give your agent eyes and ears. It can capture screenshots, record your screen with voice, and receive structured reports mid-conversation.

Claude Code (~/.claude/settings.json), Cursor, and Windsurf all take the same shape:

{
  "mcpServers": {
    "MarkuprPlus": {
      "command": "npx",
      "args": ["--yes", "--package", "markuprplus", "markuprplus-mcp"]
    }
  }
}

Tools

You: "The sidebar is overlapping the main content on mobile. Can you see it?"

Agent: [calls capture_screenshot]
       "I can see it — the sidebar is position: fixed with no z-index,
        280px wide with no responsive breakpoint. Fixing the CSS..."

Full MCP documentation: README-MCP.md.

CLI

npx markuprplus analyze ./recording.mov

markuprplus analyze ./recording.mov --output ./reports
markuprplus analyze ./recording.mov --template github-issue
markuprplus analyze ./recording.mov --no-frames        # transcript only
markuprplus watch ~/Desktop --output ./reports
markuprplus push github ./report.md --repo myorg/myapp --dry-run

Output templates: markdown (default) · json · github-issue · linear · jira The desktop app also exports html and pdf.

Requirements: Node.js 20.9+ and ffmpeg on your PATH (brew install ffmpeg / apt install ffmpeg / choco install ffmpeg).

Integrations

GitHub Action

Analyze recordings in CI and post structured feedback on the pull request that needs it:

- uses: eddiesanjuan/markuprx-action@v1
  with:
    video-path: ./recordings/
    github-token: ${{ secrets.GITHUB_TOKEN }}
    create-issues: 'true'

See markuprx-action/README.md for every input.

Issue trackers

Push each finding straight into GitHub Issues or Linear — screenshot, narration, timestamp, and capture context already formatted — from the app, the CLI, or your agent over MCP.

Keyboard Shortcuts

Recording, screenshot, and pause are global and rebindable in Settings → Hotkeys. Full reference: docs/KEYBOARD_SHORTCUTS.md.

How It Works

                    +-----------+
  Screen + Voice -> | Whisper   | -> Timestamped transcript
                    +-----------+
                         |
                    +-----------+
                    | Aligner   | -> Marks matched to the words around them
                    +-----------+
                         |
                    +-----------+
                    | Provider  | -> Structure and severity (or local rules)
                    +-----------+
                         |
                    +-----------+
                    | Generator | -> Markdown, HTML, JSON, PDF, or tracker-ready
                    +-----------+

The pipeline degrades gracefully at every step. No ffmpeg? Transcript-only output. No Whisper model? Timer-based screenshots. No provider? Local rules. A failure anywhere still leaves you a report and the raw session on disk.

For architecture details, see CLAUDE.md.

Development

npm install
npm run dev

Contributing

1. Fork the repository

2. Create a feature branch: git checkout -b feature/your-feature

3. Run tests: npm test && npm run lint && npm run typecheck

4. Open a Pull Request

See CONTRIBUTING.md for full guidelines.

License

MIT — see LICENSE.