# claude-screen-mcp

An MCP server that lets a model look at your screen: screenshots, OCR, and
change detection. It has no tools for clicking, typing, or moving the mouse.

[![CI](https://github.com/ssh071102-code/claude-screen-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/ssh071102-code/claude-screen-mcp/actions/workflows/ci.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Node](https://img.shields.io/badge/node-%3E%3D20-brightgreen.svg)](package.json)

## What it looks like

Excerpt from the e2e wire suite (`npm run test:e2e`) driving the server over
MCP JSON-RPC against a live Windows desktop:

```text
init server: { name: 'claude-screen-mcp', version: '0.4.1' }
tools: screenshot, screenshot_region, list_displays, list_windows, read_screen_text, find_text_on_screen, screenshot_if_changed, get_screen_diff, wait_for_change, record_screen

read_screen_text: 164 chars OCR'd
find_text_on_screen: 0 matches for "the"

--- Smart vision-diff (perceptual hash) ---
get_screen_diff (first): cacheKey=primary | reason=no_baseline | distance=n/a | previous_age_ms=n/a | dhash=b6a2e1b3bba3e3b3 | baseline_updated=true
get_screen_diff (second): cacheKey=primary | reason=ok | distance=0 | previous_age_ms=629 | dhash=b6a2e1b3bba3e3b3 | baseline_updated=false
screenshot_if_changed (first, no baseline): hasImage=true
screenshot_if_changed (second, threshold=64): hasImage=false (expected false — same screen)

--- wait_for_change (3s timeout, expect timeout on idle screen) ---
wait_for_change: outcome=timeout elapsed=3587ms

--- record_screen (2s @ 2fps) ---
record_screen: captured=4 returned=4 (image content blocks=4)

E2E OK (all 10 tools)
```

## Quick start

Requires Node.js 20 or newer.

Install from source:

PowerShell (Windows):

```powershell
git clone https://github.com/ssh071102-code/claude-screen-mcp
cd claude-screen-mcp
npm ci
npm run build

claude mcp add screen -- node "$PWD\dist\index.js"
```

bash (macOS / Linux):

```bash
git clone https://github.com/ssh071102-code/claude-screen-mcp
cd claude-screen-mcp
npm ci
npm run build

claude mcp add screen -- node "$(pwd)/dist/index.js"
```

> **Note on npm:** the version currently on npm is 0.4.0, which predates the
> 0.4.1 fixes listed in [CHANGELOG.md](CHANGELOG.md). Until 0.4.1 is published,
> install from source as above. `claude mcp add screen -- npx -y claude-screen-mcp`
> works but runs 0.4.0.

Restart the MCP host after registering. To check the whole stack against
your own desktop from a source checkout:

```bash
npm run test:e2e
```

## Tools

| Tool | Purpose |
|---|---|
| `screenshot` | Capture a full display and resize the image result. |
| `screenshot_region` | Capture a rectangular region. |
| `list_displays` | Enumerate connected displays. |
| `list_windows` | List visible top-level windows with optional title filter. |
| `read_screen_text` | Run OCR on the full display or a region. |
| `find_text_on_screen` | Search OCR text and return matching bounding boxes. |
| `screenshot_if_changed` | Capture only when perceptual-hash distance exceeds a threshold. |
| `get_screen_diff` | Return hash-distance diagnostics without an image. |
| `wait_for_change` | Poll until the screen changes or a timeout elapses. |
| `record_screen` | Sample a short interval and return deduplicated keyframes. |

Invalid arguments come back as a tool result with `isError: true` and the
failing field named, so the model can correct the call and retry.

## Design notes

**Read-only.** The server has no click, type, or input tool. If the model
misreads the situation, the worst it can do with this server is take a
screenshot.

**OCR first.** For monitoring, the text on screen is usually what matters.
`read_screen_text` and `find_text_on_screen` return text, which is normally
much smaller than an image in the model's context. Use `screenshot` when
layout matters.

**Perceptual-hash change detection.** `screenshot_if_changed`,
`get_screen_diff`, and `wait_for_change` share a 64-bit dHash pipeline
(greyscale, 9x8 resize, adjacent-pixel compare) with SWAR popcount for
hamming distance and an LRU baseline cache (256 entries, 24h stale TTL).
When the screen has not changed, these tools return text diagnostics and no
image.

**No real-time video.** MCP is request/response and every tool call costs a
model turn (roughly 1-3 s end to end), so streaming video does not fit. The
substitutes are `wait_for_change` (block until something happens) and
`record_screen` (sample a window of activity and return the informative
keyframes).

## Platform support

Windows 10+ is the primary development target. CI builds on Ubuntu, Windows,
and macOS with Node 20 and 22 and runs the pure-logic unit suite on all of
them. The full 10-tool MCP wire e2e (`npm run test:e2e`) needs a real display
and is run on Windows before each release. The macOS and Linux capture paths
are implemented but less tested; bug reports are welcome.

Window enumeration needs platform tooling: PowerShell (bundled) on Windows,
System Events on macOS, `wmctrl` on Linux/X11. Multi-monitor display
enumeration is currently Windows-only.

### macOS permissions

Grant the MCP host app **Screen Recording** permission (System Settings >
Privacy & Security) or captures come back black. Window listing also needs
the **Automation** permission for System Events; macOS prompts on first use.

## Security and privacy

All processing is local. No screenshot, OCR text, or telemetry leaves the
machine. The only network call is the first-run download of Tesseract
language data.

OCR output is untrusted input. Text on screen may try to influence the
model. Treat it as user-supplied data and do not auto-execute commands
derived from it. Scope `read_screen_text` to a region when you do not need
the whole desktop.

## Configuration

| Variable | Default | Purpose |
|---|---|---|
| `SCREEN_MCP_LOG_LEVEL` | `info` | `debug`, `info`, `warn`, or `error`. |
| `SCREEN_MCP_OCR_LANGS` | `eng+chi_sim` | Tesseract language list (allowlist enforced). |

The first OCR call downloads language data (about 7.5 MB for the default
`eng+chi_sim`, measured on disk). Later calls reuse the local cache.

## Performance

`npm run validate` runs the benchmark below and fails if a P95 budget is
exceeded. Numbers from one run on a Windows 11 laptop (2026-09-27); expect
variation with screen size and machine load.

| Operation | P50 | P95 | P95 budget |
|---|---|---|---|
| Full-screen capture + resize + JPEG (maxEdge 1600) | 545 ms | 589 ms | 2000 ms |
| Raw full-screen PNG capture | 674 ms | 874 ms | 3000 ms |
| 800x600 region capture (PNG) | 530 ms | 833 ms | 2000 ms |
| dHash of a full screen | 39 ms | 60 ms | 300 ms |

## Limitations

- **Every capture on Windows starts a new `powershell.exe`**, which costs
  about 0.5 s (see the table above), so each sample is a short-lived
  PowerShell process. This affects the two sampling tools differently:
  - `wait_for_change` sleeps `pollMs` and then captures, so on Windows one
    poll takes `pollMs` plus about 0.5 s. With the default `pollMs=500` that
    is roughly one sample per second.
  - `record_screen` takes `targetFps` (default 2, min interval 200 ms) and
    schedules frame *n* at `n / targetFps` seconds from the start. It sleeps
    only for whatever is left until that deadline, so capture time is not
    added on top of the interval. If a capture overruns its slot, the next one
    starts immediately. On Windows the ~0.5 s capture caps the real rate at
    about 2 fps whatever `targetFps` asks for.
- **`list_windows` on Windows takes about 1 s**, because each call compiles a
  small C# shim (`EnumWindows`) inside PowerShell.
- **Mixed-DPI multi-monitor on Windows**: captures of secondary displays may
  come back scaled when monitors have different DPI factors.
- **Window handles are platform-dependent**: Windows returns the HWND and
  Linux/X11 the X11 window id, both stable while the window exists. On macOS
  the handle is the position in that listing and changes as windows open
  and close, so list again before relying on it.
- **OCR quality depends on display scaling**: small text at 100% scaling on
  high-DPI screens OCRs poorly. Capture a region or increase scaling.
- **No input control**: this server cannot click, type, or move the mouse.
  If you pair it with an input-capable tool, that tool's risks are yours.

## Development

```bash
npm ci
npm run build
npm test           # pure-logic unit tests (no display needed; runs in CI)
npm run test:e2e   # full 10-tool MCP wire test (needs a live desktop)
npm run validate   # perf benchmark, fails on exceeded P95 budgets
```

## Roadmap

- `screenshot_window(title)` for direct single-window capture.
- Multi-display enumeration on macOS and Linux.

## AI assistance
This project was built with AI assistance (Claude Code). Commits it co-authored carry a `Co-Authored-By` trailer.

## License

MIT, see [LICENSE](LICENSE).