# Screen Capture MCP

An MCP (Model Context Protocol) server that gives AI agents eyes on your screen. Define a capture region with a visual overlay, capture any window by name, or grab arbitrary coordinates — and let your agent screenshot it on demand.

Built for [Claude Code](https://docs.anthropic.com/en/docs/claude-code) but works with any MCP-compatible client. macOS only.

---

## How It Works

Two components work together:

1. **Overlay** (`overlay.py`) — A transparent, draggable frame you position over any window. It saves its region (plus its PID, so the server knows it's alive) to a state file.
2. **MCP Server** (`server.js`) — Reads that region and uses macOS `screencapture` to grab it, returning a base64 PNG to the agent. It can also capture windows by name and arbitrary regions without the overlay.

```
┌──────────────────────────┐
│  Your App Window         │
│  ┌────────────────────┐  │
│  │ ▓▓ Overlay Frame ▓▓│  │      overlay.py
│  │ ▓                 ▓│  │   saves region + PID
│  │ ▓  (capture area) ▓│──────▶ viewport.json
│  │ ▓                 ▓│  │
│  │ ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓│  │
│  └────────────────────┘  │
│                          │      server.js
└──────────────────────────┘   reads region + screencapture
                                      │
                                      ▼
                               base64 PNG → Agent
```

---

## Tools

The MCP server exposes five tools:

| Tool | Description |
|---|---|
| `capture_viewport` | Capture the overlay region. Returns the image + metadata. |
| `get_viewport_info` | Read the overlay coordinates and whether the overlay is running. |
| `capture_region` | Capture any screen region by x, y, width, height. No overlay needed. |
| `capture_window` | Capture a window by title/app name match (e.g. `"simulator"`). No overlay needed, follows the window if it moves. |
| `list_windows` | List visible windows with app, title, id, position and size. |

All capture tools accept an optional `full_resolution: true` to return the native Retina-resolution image. By default images are downscaled to point dimensions — on a 2x display that's a quarter of the pixels (and tokens) with no practical loss for UI work.

---

## Setup

### Prerequisites

- macOS (uses native `screencapture`)
- Node.js 18+
- Python 3.8+ with Tkinter (for the overlay)

### Install

```bash
git clone https://github.com/Sherif-Aboulnasr/Agent-Screen-Capture-MCP.git
cd Agent-Screen-Capture-MCP
npm install
```

### Grant Screen Recording permission

macOS requires Screen Recording permission for the **app that launches the MCP server** — your terminal (Terminal, iTerm2) or editor (VS Code):

**System Settings → Privacy & Security → Screen & System Audio Recording** → enable your terminal/editor, then restart it.

> ⚠️ Without this permission `screencapture` does **not** error — it silently captures only your wallpaper. If the agent reports captures that are one flat color or show no app windows, this permission is the first thing to check. (The server detects flat-color captures and warns the agent.)

### Configure Claude Code

```bash
claude mcp add screen-capture --scope user -- node /absolute/path/to/Agent-Screen-Capture-MCP/server.js
```

Or add it per-project in a `.mcp.json` at the project root:

```json
{
  "mcpServers": {
    "screen-capture": {
      "command": "node",
      "args": ["/absolute/path/to/Agent-Screen-Capture-MCP/server.js"]
    }
  }
}
```

For other MCP clients, use the same `command`/`args` shape in that client's MCP server config.

---

## Usage

### 1. Start the overlay

```bash
python3 overlay.py
```

A lime-green border frame appears on screen. The top bar shows the current capture size (e.g. `374×793`).

### 2. Position it

- **Drag** the top bar to move the frame over your target window
- **Drag** the left/right/bottom edges or the bottom-right corner handle to resize
- **Arrow keys** nudge 1px, **Alt+arrows** 10px, **Shift+arrows** resize — click the top bar first so the overlay has keyboard focus
- **Click the lock icon** to lock the frame (border turns red) — this prevents accidental moves; capture works whether locked or not

### 3. Let your agent capture

The agent can call `capture_viewport` at any time to see what's inside the frame. The border flashes white for a moment on every capture, so you always know when the agent looked.

```
Agent: "Let me check how that looks..."
→ calls capture_viewport
→ receives screenshot of the framed region
→ "The button is misaligned — let me fix the padding..."
```

If you quit the overlay, the server notices (the saved PID is no longer alive) and refuses to capture a stale region — the agent is told to ask you to restart it. Your frame position is remembered for the next launch.

### Capture without the overlay

Capture a window by name — no overlay positioning needed, and it follows the window if it moves:

```json
{ "title": "simulator" }
```

Or grab exact coordinates with `capture_region`:

```json
{ "x": 100, "y": 200, "width": 800, "height": 600 }
```

---

## Overlay controls

| Control | Action |
|---|---|
| Top bar drag | Move the frame |
| Left / right / bottom edge drag | Resize that edge |
| Bottom-right corner handle | Resize freely (sits outside the frame, never appears in captures) |
| Arrow keys / Alt+arrows | Nudge 1px / 10px |
| Shift+arrows / Shift+Alt+arrows | Resize 1px / 10px |
| Lock icon (top-left) | Toggle lock (prevents accidental moves) |
| X button (top-right) | Quit overlay |
| ESC | Quit overlay (needs keyboard focus — click the top bar first if it doesn't respond) |

| State | Border color |
|---|---|
| Unlocked (moveable) | Lime green |
| Locked | Red |
| Capture just happened | Brief white flash |

---

## State file

The overlay writes its region to:

```
~/Library/Application Support/screen-capture-mcp/viewport.json
```

Override with the `SCREEN_CAPTURE_VIEWPORT_FILE` environment variable — set it identically for **both** the server and the overlay. The file includes the overlay's PID and a timestamp; the server uses the PID to verify the overlay is actually running before trusting the coordinates. Writes are atomic, and the overlay refuses to launch if another instance is already running.

---

## How it's built

- **Overlay**: Python Tkinter — four separate edge windows with a 3px border, keeping the center click-through. The resize handle is a fifth window hanging outside the bottom-right corner so it never contaminates captures.
- **Server**: Node.js using `@modelcontextprotocol/sdk`. Reads the state file, verifies overlay liveness by PID, runs `screencapture`, downscales Retina output to point size via `sips`, returns base64 PNG. Window capture resolves window IDs via a small JXA (`osascript`) call to `CGWindowListCopyWindowInfo`.
- **Capture flash**: the server bumps a signal file after each capture; the overlay polls its mtime and flashes the border.

### Notes & limitations

- **Multi-monitor**: coordinates are only verified on the primary display. On secondary displays (especially ones arranged above/left of the main, or with a different scale factor) the overlay position and `screencapture`'s coordinate space may not line up — keep the frame on the primary display.
- **Window titles** in `list_windows` / `capture_window` require the same Screen Recording permission; without it they come back empty.

---

## License

MIT — see [LICENSE](LICENSE).