# screen-mcp

Screen-aware MCP (Model Context Protocol) server. Captures the user's screen, finds text-worthy regions via heuristic analysis, runs OCR, and returns a compact JSON (~500 tokens) instead of a full screenshot (~500,000 tokens).

Architecture

Related MCP server: MCP Screen Text

Three Modes

1. MCP stdio (for Codex / Claude Desktop / Cursor)

Add to your MCP config (mcp.json):

json { "mcpServers": { "screen": { "command": "python", "args": ["-m", "src.main"], "cwd": "/path/to/screen-mcp" } } }

Three tools are exposed:

• screen_analyze - Full analysis: capture -> detect regions -> OCR

• screen_watch - Background monitoring: start / stop / status / force_check

• screen_diff - One-shot change detection

2. Demo Mode

ash python -m src.main --demo

Outputs a JSON analysis of the current screen to stdout.

3. HTTP Debug Server

ash python -m src.main --http-port 8000

• GET /health returns {"status": "ok"}

• POST /analyze returns full screen analysis JSON

Installation

`ash pip install mss Pillow pytesseract numpy fastapi uvicorn

Install Tesseract OCR separately

Windows: winget install UB-Mannheim.TesseractOCR

macOS: brew install tesseract

Linux: sudo apt install tesseract-ocr

`

Output Example

json { "monitors": [{"left": 0, "top": 0, "width": 1920, "height": 1080}], "regions": [{"x1": 420, "y1": 140, "x2": 1160, "y2": 460, "confidence": 0.76}], "texts": [ {"text": "New chat", "confidence": 0.96, "bbox": [43, 54, 100, 64]}, {"text": "Search", "confidence": 0.96, "bbox": [43, 84, 82, 94]} ], "summary": "Found 5 text regions | Text (118): New chat, Search, Plugins..." }

Token Savings

Configuration

Notes

• OCR uses Tesseract with English language pack by default

• Chinese OCR requires installing chi_sim.traineddata

• The ScreenWatcher background monitor uses MSE-based frame differencing with configurable interval and threshold

• Tesseract auto-detects common installation paths; falls back to PATH lookup