# clipboard-vision-mcp

> 🇫🇷 **[Version française disponible → README.fr.md](README.fr.md)**

> Add vision to text-only models in Opencode (**DeepSeek V4**, **GLM 5.3**) — **see the image in your clipboard directly**, no manual file saving.

**Tested on Windows 11 + Opencode + DeepSeek V4 Pro.** Multi-OS clipboard support (Windows / macOS / Linux X11 / Linux Wayland).

Forked from [itcomgroup/vision-mcp-server](https://github.com/itcomgroup/vision-mcp-server) — rewritten around clipboard-first tools, security hardening, cross-platform clipboard extraction, and one-prompt AI install.

---

## The problem

Cheap/fast text-only models like **DeepSeek V4** and **GLM 5.3** are great for code, but they cannot read images. Every time you paste a screenshot, the model asks you to save it to disk and provide a path.

## The fix

This MCP server exposes `*_from_clipboard` tools. When the LLM needs to see your screenshot, it calls `analyze_clipboard` — the server reads the clipboard image, sends it to a real vision model (**Groq + Qwen 3.6 27B, free tier**), and returns a text description the text model can reason about.

Result: **paste → ask → done.** No file shuffling.

---

## 🤖 One-prompt install via AI (recommended)

Instead of running the steps below manually, paste one of these prompts into any coding assistant (DeepSeek, GLM, Claude, GPT, ...) and it will set everything up for you end-to-end — clone, venv, deps, MCP config, keybindings:

- 🇬🇧 **[docs/INSTALL_PROMPT_EN.md](docs/INSTALL_PROMPT_EN.md)**
- 🇫🇷 **[docs/INSTALL_PROMPT_FR.md](docs/INSTALL_PROMPT_FR.md)**

Prefer doing it yourself? Keep reading.

---

## Features

- 🖼️ **Clipboard-first** — `analyze_clipboard`, `extract_text_from_clipboard`, `diagnose_error_from_clipboard`, `describe_ui_from_clipboard`, `code_from_clipboard`.
- 📁 **File path fallback** — same tools available for images already on disk.
- 🆓 **Free vision backend** — Groq's free tier with Qwen 3.6 27B (27B multimodal, 131K context).
- 🔁 **Model is swappable** — set `GROQ_VISION_MODEL` to follow Groq's catalog without touching the code.
- 🖥️ **Multi-OS** — Windows, macOS, Linux (X11 + Wayland).
- 🔒 **Security hardened** — extension/size/magic-byte validation, auto-delete of clipboard temp files after analysis.
- 🔌 **MCP standard** — works with Opencode, Claude Code, Cursor, Cline, Continue, or any MCP-capable client.

---

## Requirements

- **Python 3.10+**
- **Groq API key** (free, 30 seconds sign-up): https://console.groq.com/keys
- An MCP-capable client (Opencode, Claude Code, Cursor, Cline, Continue, ...)

### Python dependencies (installed automatically via `pip install -e .`)

| Package | Purpose |
|---|---|
| `mcp>=1.0.0` | MCP protocol server |
| `groq>=0.11.0` | Groq API client (Qwen 3.6 27B vision) |
| `aiofiles>=23.0.0` | Async file I/O |
| `Pillow>=10.0.0` | Clipboard image extraction (Windows/macOS), PNG encoding |

### OS-specific clipboard dependencies

| OS | Command | Why |
|---|---|---|
| **Windows** | *nothing extra* | Pillow + pywin32 handle the clipboard natively. |
| **macOS** | `brew install pngpaste` *(optional fallback)* | Pillow works in most cases; pngpaste as backup. |
| **Linux — Wayland** | `sudo apt install wl-clipboard` | Provides `wl-paste`. |
| **Linux — X11** | `sudo apt install xclip` | Or your distro equivalent. |

---

## Quick start

### 1. Get a free Groq API key
https://console.groq.com/keys

### 2. Install

```bash
git clone https://github.com/Capetlevrai/clipboard-vision-mcp.git
cd clipboard-vision-mcp
python -m venv .venv
# Windows:
.venv\Scripts\activate
# macOS/Linux:
source .venv/bin/activate
pip install -e .
```

### 3. Smoke test (no Groq needed)

Copy any screenshot, then:

```bash
python examples/smoke_test.py
```

Expected: `OK: clipboard image saved to <path>`.

### 4. Wire to your MCP client

See [docs/OPENCODE.md](docs/OPENCODE.md) for Opencode (Windows-tested) or [docs/CLIENTS.md](docs/CLIENTS.md) for Claude Code / Cursor / Cline / Continue.

**Opencode** (`%APPDATA%\opencode\opencode.json` on Windows, `~/.config/opencode/opencode.json` on Linux/macOS):

```json
{
  "mcp": {
    "clipboard-vision": {
      "type": "local",
      "command": [
        "C:\\path\\to\\clipboard-vision-mcp\\.venv\\Scripts\\python.exe",
        "-m",
        "clipboard_vision_mcp"
      ],
      "enabled": true,
      "environment": {
        "GROQ_API_KEY": "gsk_your_key_here"
      }
    }
  }
}
```

> 💡 **Use the absolute path to the venv's Python.** This guarantees the MCP starts with the right dependencies regardless of shell, cwd, or active venv.

### 4b. Changing the vision model (optional)

The model id is read once at startup from `GROQ_VISION_MODEL`, defaulting to `qwen/qwen3.6-27b`. Add it to the same `environment` block to pin a different one:

```json
"environment": {
  "GROQ_API_KEY": "gsk_your_key_here",
  "GROQ_VISION_MODEL": "qwen/qwen3.6-27b"
}
```

Useful when Groq retires a model — as happened to `meta-llama/llama-4-scout-17b-16e-instruct` on 2026-06-17. Pick any vision model from https://console.groq.com/docs/models; no code change required. The older `VISION_MODEL` name still works as a fallback.

### 5. ⚠️ Opencode keybindings for pasting images (Alt+V)

Two important facts about opencode (these are easy to get wrong):

- Keybinds live in **`tui.json`** (next to `opencode.json`), **NOT** in `opencode.json`. `opencode.json` rejects unknown top-level keys, so adding a `keybinds` block there makes opencode **refuse to start** (`ConfigInvalidError`).
- There is **no** `input_paste_image` action. Pasting (text **and** images) is the single `input_paste` action — opencode's TUI reads the image straight from the OS clipboard. It is bound to `Ctrl+V` by default; to also paste with `Alt+V`, bind `input_paste` to both.

Create/merge `tui.json` (e.g. `~/.config/opencode/tui.json` on Linux/macOS, `C:\Users\<you>\.config\opencode\tui.json` on Windows):

```json
{
  "$schema": "https://opencode.ai/tui.json",
  "keybinds": {
    "input_paste": [
      { "key": "ctrl+v", "preventDefault": false },
      { "key": "alt+v", "preventDefault": false }
    ]
  }
}
```

Restart Opencode after editing. Then copy a screenshot and press `Alt+V` (or `Ctrl+V`) in the prompt — the image attaches.

> ℹ️ Optional: the `*_from_clipboard` tools read the OS clipboard themselves, so you can skip pasting entirely — just copy a screenshot and ask the model to "analyze my clipboard".

### 6. Does it auto-start after a Windows reboot?

**Yes.** Opencode re-reads `opencode.json` at every launch and auto-spawns any MCP server with `"type": "local"` and `"enabled": true`. Because the command uses the **absolute path to the venv's Python**, it doesn't matter which shell or working directory Opencode is launched from.

Reboot → open Opencode → `clipboard-vision` tools are listed. No manual step.

---

## Usage

```
You: (copy a screenshot to clipboard, then type)
     "Look at what I just copied and tell me what's wrong with this error."

LLM (DeepSeek, GLM, Claude, ...): [calls diagnose_error_from_clipboard]
     → "The error says `ECONNREFUSED 127.0.0.1:5432`. Postgres isn't
        running on port 5432. Start it with: ..."
```

The text-only model never sees pixels — it reads the description returned by Qwen 3.6 27B and reasons over it.

### Tool reference

| Tool | Input | Use when |
|---|---|---|
| `analyze_clipboard` | optional `prompt` | Generic description, Q&A on the clipboard image. |
| `extract_text_from_clipboard` | — | Pure OCR. |
| `describe_ui_from_clipboard` | — | UI/UX review, component inventory. |
| `diagnose_error_from_clipboard` | — | Error screenshot → cause + fix. |
| `code_from_clipboard` | — | Extract code from a screenshot. |
| `analyze_image` | `image_path`, optional `prompt` | Image already on disk. |
| `extract_text`, `describe_ui`, `diagnose_error`, `understand_diagram`, `analyze_chart`, `code_from_screenshot` | `image_path` | Same as above for files. |

---

## Security

This server runs as a **local stdio process** — it does not open any network port and only talks to the MCP client over stdin/stdout and to the Groq API over HTTPS.

Hardening in place:

- **File type allow-list.** `analyze_image` and the other file-path tools only accept `.png .jpg .jpeg .gif .webp .bmp`. This prevents a prompt-injected LLM from asking the server to read arbitrary local files (`~/.ssh/id_rsa`, `.env`, ...) and exfiltrate them as base64 to Groq.
- **Magic-byte check.** File content is validated against known image headers before upload.
- **Size cap.** 20 MB max per image.
- **Auto-delete clipboard temp files** after each analysis. Screenshots may contain secrets (tokens, chats, credentials) — the server writes them to `$TMPDIR/clipboard_vision_mcp/` and unlinks them on completion.
- **No telemetry.** No analytics, no phone-home.

### What this project cannot protect you from

- **Your API key lives in your MCP client config in plain text.** That is how MCP clients work today. Keep that config file non-world-readable and never commit it. If you accidentally expose a key (chat, screenshot, git push), **rotate it** at https://console.groq.com/keys.
- **Groq receives the images you analyze.** Check their [privacy policy](https://groq.com/privacy-policy/) before sending anything sensitive.
- **Any MCP tool is executed at the direction of the LLM.** If you connect a prompt-injected model to this server and feed it untrusted input, the model can choose what to analyze. The allow-list above reduces blast radius but cannot eliminate it.

### Found an issue?

Please open a [private security advisory](https://github.com/Capetlevrai/clipboard-vision-mcp/security/advisories/new) rather than a public issue.

---

## How it works

```
┌──────────────┐   MCP   ┌─────────────────┐   HTTPS   ┌─────────────────┐
│  Opencode    │ ──────▶ │  clipboard-     │ ────────▶ │  Groq API       │
│  (DeepSeek)  │         │  vision-mcp     │           │  Qwen 3.6 27B   │
└──────────────┘         └─────────────────┘           └─────────────────┘
                              │
                              ▼
                     reads system clipboard
                     (PIL / wl-paste / xclip)
                     → validate → base64 → send → delete
```

---

## Troubleshooting

- **"Clipboard does not contain an image."** — Copy an actual image, not a file icon or text. On Linux, verify `wl-paste --type image/png` or `xclip -selection clipboard -t image/png -o | file -` works outside the MCP.
- **"GROQ_API_KEY is not set."** — Check the `environment` block in your client config, then **fully restart** the client.
- **Tools don't appear in Opencode.** — Check Opencode's MCP logs. Run `python -m clipboard_vision_mcp` manually — it should start and wait silently on stdin.
- **"Refusing to read '<ext>' — only image files are allowed."** — You (or the LLM) passed a non-image path. This is the security guard doing its job.
- **`model_not_found` / `model has been decommissioned`.** — Groq retired the configured vision model (this is what killed `meta-llama/llama-4-scout-17b-16e-instruct` on 2026-06-17). Set `GROQ_VISION_MODEL` to a current id from https://console.groq.com/docs/models and restart the client.

---

## Credits

- Forked from [itcomgroup/vision-mcp-server](https://github.com/itcomgroup/vision-mcp-server) — original Groq + Llama-4 Scout MCP integration.
- Vision model: [Qwen 3.6 27B](https://groq.com/) served by Groq (replaces Llama-4 Scout 17B, deprecated 2026-06-17).

## License

MIT — see [LICENSE](LICENSE).