eyes-mcp

Give any text-only LLM eyes. Local vision for your coding agent.

One command · zero API keys · nothing leaves your machine

DeepSeek, GLM, Qwen-Coder, Llama… great models, all blind.

❌ Without eyes

You paste a screenshot into your agent (running a text-only model via Claude Code / Codex / Cursor):

> Here's the error in my UI, fix it  [screenshot.png]

I'm sorry — I cannot see images. Please describe the error in text.

✅ With eyes

The agent calls a local VLM + OCR instead, and reads the screenshot itself:

> Here's the error in my UI, fix it  [screenshot.png]

I see a React hydration error in `CartDrawer.tsx:142`. The OCR shows:
"Hydration failed because the server rendered HTML didn't match the client." …

Quickstart

git clone https://github.com/JamesbbBriz/eyes-mcp
cd eyes-mcp && ./scripts/install.sh

That's it. The installer:

1. asks which model you want, with a recommendation computed from your RAM and GPU (skip the question with EYES_PRESET or --yes),

2. installs deps and downloads the model (~0.3 to 3.5GB, resumable),

3. detects which of your agents run text-only models, by reading your Claude Code / Codex / Cursor configs and checking each model against a modality database,

4. registers eyes-mcp only where it's needed. Multimodal agents are skipped automatically.

# options:
EYES_PRESET=fast ./scripts/install.sh            # Qwen3.5-0.8B, natively multimodal
HF_ENDPOINT=https://hf-mirror.com ./install.sh  # mainland-CN mirror
./install.sh --yes                              # accept all recommendations, no prompts
./install.sh --dry-run                          # preview without changing anything

Just want the modality check? python3 scripts/detect_modality.py

Requires: Python ≥3.11, llama.cpp (brew install llama.cpp), ~1GB RAM.

Manual registration

Skipped auto-install, or an agent the installer doesn't know? Add it by hand.

Claude Code (~/.claude.json → mcpServers):

"eyes-mcp": {
  "command": "uv",
  "args": ["--directory", "/ABS/PATH/eyes-mcp", "run", "eyes-mcp"],
  "env": { "EYES_PRESET": "lfm-450m" }
}

Codex (~/.codex/config.toml):

[mcp_servers.eyes-mcp]
command = "uv"
args = ["--directory", "/ABS/PATH/eyes-mcp", "run", "eyes-mcp"]
env = { EYES_PRESET = "lfm-450m" }

Cursor (.cursor/mcp.json): same shape as Claude Code.

Restart the agent, then ask: "what's in this screenshot?"

Tools

Model presets

Hidden extras (still one command): smol500 (SmolVLM2-500M), paddle (PaddleOCR-VL-1.6), qwen3-2b (Qwen3-VL-2B).

Any other GGUF works too. Point the env at it and skip presets entirely:

EYES_MODEL_DIR=~/models/my-vlm  VLM_MODEL_FILE=model-Q4.gguf  VLM_MMPROJ_FILE=mmproj.gguf

Good candidates not shipped as presets: LFM2.5-VL-1.6B/3B, InternVL3.5-2B/4B, MiniCPM-V-4.6, DeepSeek-OCR, dots.ocr, gemma-3n-E2B, moondream2. Anything llama.cpp supports with an mmproj file works.

Switch anytime: set EYES_PRESET and run ./scripts/download_models.sh again. Not sure which? python3 scripts/choose_model.py shows your RAM/GPU and marks a recommendation.

How it works

Claude Code / Codex / Cursor
      │ MCP stdio
      ▼
eyes-mcp  (stateless, mcp SDK 2.x)
   ├─ analyze_image → llama.cpp llama-server (local VLM)  "understand"
   └─ ocr_image     → RapidOCR (onnx, ~20MB)             "extract text"

• Lifecycle follows your agent: the VLM server spawns when the MCP starts and shuts down when your agent exits, so you never end up with orphan processes or a daemon to babysit.

• Floating port: the VLM never binds a fixed port (goodbye, "8080 already in use"), so it coexists with your other local services.

• External VLM reuse: if you already run one at VLM_BASE_URL, eyes-mcp uses it instead of spawning its own.

Why

The cheapest and best coding models right now (DeepSeek-V4-Flash, GLM-5.x, Qwen-Coder) are text-only. Every harness assumes you can paste a screenshot, and every one of these models silently fails at it. eyes-mcp is the missing sidecar: a small local VLM plus OCR, wrapped in the lifecycle your agent already understands.

Roadmap

• Lazy VLM start (spawn on first tool call, not MCP start)

• screenshot_analyze (grab the screen, no file needed)

• PDF pages → vision

• npx eyes-mcp one-liner installer

• Per-model prompt templates (llama.cpp OCR models need specific prompts)

FAQ

Does my agent model matter? Only in that it must be text-only for this to be useful. Multimodal models (GPT, Claude, GLM-V) already see images, so don't bother.

GPU needed? No. It runs fine on CPU, and llama.cpp picks up Apple Metal or CUDA automatically when present.

Where are models stored? ~/.eyes-mcp/models/<preset>/. Delete them to reset.

License

MIT. Model weights keep their own licenses (see preset table); they're downloaded at install time, never redistributed here.