# claude-screen-mcp

An MCP server that exposes read-only screen state. Capture, OCR, and change
detection only — no input control.

[![CI](https://github.com/ssh071102-code/claude-screen-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/ssh071102-code/claude-screen-mcp/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Node](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](package.json)

## What it looks like

Excerpt from the e2e wire suite (`npm run test:e2e`) driving the server over
real MCP JSON-RPC against a live desktop:

```text
init server: { name: 'claude-screen-mcp', version: '0.4.0' }
tools: screenshot, screenshot_region, list_displays, list_windows, read_screen_text, find_text_on_screen, screenshot_if_changed, get_screen_diff, wait_for_change, record_screen

read_screen_text: 70 chars OCR'd
find_text_on_screen: 0 matches for "the"

--- Smart vision-diff (perceptual hash) ---
get_screen_diff (first): cacheKey=primary | reason=no_baseline | distance=n/a | previous_age_ms=n/a | dhash=ecaf878c8c858e83 | baseline_updated=true
get_screen_diff (second): cacheKey=primary | reason=ok | distance=0 | previous_age_ms=477 | dhash=ecaf878c8c858e83 | baseline_updated=false
screenshot_if_changed (first, no baseline): hasImage=true
screenshot_if_changed (second, threshold=64): hasImage=false (expected false — same screen)

--- wait_for_change (3s timeout, expect timeout on idle screen) ---
wait_for_change: outcome=timeout elapsed=3575ms

--- record_screen (2s @ 2fps) ---
record_screen: captured=4 returned=4 (image content blocks=4)

E2E OK (all 10 tools)
```

## Quick start

PowerShell (Windows):

```powershell
git clone https://github.com/ssh071102-code/claude-screen-mcp
cd claude-screen-mcp
npm install
npm run build

claude mcp add screen -- node "$PWD\dist\index.js"
```

bash (macOS / Linux):

```bash
git clone https://github.com/ssh071102-code/claude-screen-mcp
cd claude-screen-mcp
npm install
npm run build

claude mcp add screen -- node "$(pwd)/dist/index.js"
```

Restart the MCP host after registration. To verify the full stack against
your own desktop:

```bash
npm run test:e2e
```

## Tools

| Tool | Purpose |
|---|---|
| `screenshot` | Capture a full display and resize the image result. |
| `screenshot_region` | Capture a rectangular region. |
| `list_displays` | Enumerate connected displays. |
| `list_windows` | List visible top-level windows with optional title filter. |
| `read_screen_text` | Run OCR on the full display or a region. |
| `find_text_on_screen` | Search OCR text and return matching bounding boxes. |
| `screenshot_if_changed` | Capture only when perceptual-hash distance exceeds a threshold. |
| `get_screen_diff` | Return hash-distance diagnostics without an image. |
| `wait_for_change` | Poll until the screen changes or a timeout elapses. |
| `record_screen` | Sample a short interval and return deduplicated keyframes. |

## Design notes

**Read-only by design.** There is no click, type, or input tool, and there
never will be. The blast radius of a confused model holding this server is a
screenshot, not a click.

**OCR-first.** For screen monitoring, text is what you usually need, and text
tokens are 10-100x cheaper than vision tokens. `read_screen_text` and
`find_text_on_screen` let the model read the screen without paying for
images; images are reserved for when layout actually matters.

**Perceptual-hash change detection.** `screenshot_if_changed`,
`get_screen_diff`, and `wait_for_change` share a 64-bit dHash pipeline
(greyscale, 9x8 resize, adjacent-pixel compare) with SWAR popcount for
hamming distance and an LRU baseline cache (256 entries, 24h stale TTL).
Static screens cost zero image tokens, which is what makes 24/7 watching
affordable.

**No real-time video, on purpose.** MCP is request/response and every tool
call costs an LLM turn (roughly 1-3s end to end), so 24fps streaming is
physically impossible at that latency. The practical substitutes are
`wait_for_change` (block until something happens) and `record_screen`
(sample a window of activity, return only the informative keyframes).

## Platform support

Windows 10+ is the primary development target. CI builds on Ubuntu, Windows,
and macOS and runs the pure-logic unit suite on all three; the full 10-tool
MCP wire e2e (`npm run test:e2e`) needs a real display and is run on Windows
before each release. macOS and Linux capture paths are implemented but
best-effort — bug reports welcome.

Window enumeration needs platform tooling: PowerShell (bundled) on Windows,
System Events on macOS, `wmctrl` on Linux/X11. Multi-monitor display
enumeration is currently Windows-only.

### macOS permissions

Grant the MCP host app **Screen Recording** permission (System Settings >
Privacy & Security) or captures come back black. Window listing additionally
requires the **Automation** permission for System Events; macOS prompts on
first use.

## Security and privacy

All processing is local. No screenshot, OCR text, or telemetry leaves the
machine; the only network call is the initial Tesseract language data
download.

OCR output is untrusted input. Text rendered on screen may attempt to
influence the model. Treat output as user-supplied data and avoid
auto-executing commands derived from it. Scope `read_screen_text` to a region
when full-desktop capture is not required.

## Configuration

| Variable | Default | Purpose |
|---|---|---|
| `SCREEN_MCP_LOG_LEVEL` | `info` | `debug`, `info`, `warn`, or `error`. |
| `SCREEN_MCP_OCR_LANGS` | `eng+chi_sim` | Tesseract language list (allowlist enforced). |

The first OCR call downloads language data (~7.5MB for the default
`eng+chi_sim`, measured on disk); subsequent calls reuse the local cache.

## Performance

Measured on a Windows dev machine, enforced by `npm run validate`, which
fails if P95 budgets are exceeded:

| Operation | P50 | P95 | P95 budget |
|---|---|---|---|
| Full-screen capture + resize + JPEG (maxEdge 1600) | 487 ms | 516 ms | 2000 ms |
| Raw full-screen PNG capture | 500 ms | 501 ms | 3000 ms |
| 800x600 region capture (PNG) | 478 ms | 487 ms | 2000 ms |
| dHash of a full screen | 34 ms | 35 ms | 300 ms |

## Limitations

- **Mixed-DPI multi-monitor on Windows**: captures of secondary displays may
  come back scaled when monitors have different DPI factors. Known
  limitation.
- **Window handles are platform-dependent**: Windows returns a stable HWND
  and Linux/X11 a stable window id, but the macOS handle is just the
  enumeration index of that listing and shifts as windows open and close —
  re-list before relying on it.
- **OCR quality depends on display scaling**: small text at 100% scaling on
  high-DPI screens OCRs poorly; capture a region or increase scaling.
- **No input control, ever**: this server cannot click, type, or move the
  mouse, by design. Pair it with an input-capable tool at your own risk.

## Development

```bash
npm install
npm run build
npm test           # pure-logic unit tests (no display needed; runs in CI)
npm run test:e2e   # full 10-tool MCP wire test (needs a live desktop)
npm run validate   # perf benchmark, fails on exceeded P95 budgets
```

## Roadmap

- `screenshot_window(title)` for direct single-window capture.
- Improved multi-display enumeration on macOS and Linux.

## License — MIT, see [LICENSE](LICENSE).