# Vision MCP Server

MCP server for image processing via Ollama vision models (Gemma 4, Gemma 3, LLaVA...).  
Enables LLM clients without vision capability (DeepSeek, Qwen, etc.) to process images by delegating to a local vision model through Ollama.

## Features

| Tool | Description |
|------|-------------|
| `describe_image` | Describe image content (brief / detailed / exhaustive) |
| `ocr_image` | Extract text from image with language hints (vi, en, ja, zh, ko) |
| `ask_image` | Ask any question about an image with a custom prompt |
| `process_clipboard_image` | Read image directly from macOS clipboard — no file path needed |

## Requirements

- macOS (clipboard tool uses `osascript`)
- Python 3.12+
- [uv](https://docs.astral.sh/uv/) — Python package manager
- [Ollama](https://ollama.com) — local LLM runtime

## Installation

### 1. Clone the repo

```bash
git clone https://github.com/nguyenduc/vision-mcp-server.git
cd vision-mcp-server
```

### 2. Install dependencies

```bash
uv sync
```

> `uv sync` creates `.venv/` and installs all packages from `uv.lock`. No need for `pip install` or `uv init`.

### 3. Pull a vision model

```bash
ollama pull gemma4
```

Other compatible vision models: `gemma3`, `llava`, `llava-llama3`, `moondream`.

### 4. Make sure Ollama is running

```bash
ollama serve
```

Verify:

```bash
curl http://127.0.0.1:11434/api/tags
```

### 5. Test the server

```bash
uv run server.py
```

The server runs over stdio — press `Ctrl+C` to stop.

## MCP Client Configuration

### OpenCode

Add to `.opencode.json` (project-level or `~/.opencode.json`):

```json
{
  "mcpServers": {
    "vision": {
      "enabled": true,
      "type": "local",
      "command": ["uv", "run", "server.py"],
      "cwd": "/absolute/path/to/vision-mcp-server",
      "env": ["OLLAMA_BASE_URL=http://127.0.0.1:11434", "VISION_MODEL=gemma4"]
    }
  }
}
```

> **Note**: In OpenCode, `command` is an array and `env` is an array of `"KEY=VALUE"` strings, not an object.

### Claude Desktop

Add to `~/Library/Application Support/Claude/claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "vision": {
      "command": "uv",
      "args": ["run", "server.py"],
      "cwd": "/absolute/path/to/vision-mcp-server"
    }
  }
}
```

### Cursor / Windsurf / Cline

```json
{
  "mcpServers": {
    "vision": {
      "command": "uv",
      "args": ["run", "server.py"],
      "cwd": "/absolute/path/to/vision-mcp-server",
      "env": {
        "OLLAMA_BASE_URL": "http://127.0.0.1:11434",
        "VISION_MODEL": "gemma4"
      }
    }
  }
}
```

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `OLLAMA_BASE_URL` | `http://127.0.0.1:11434` | Ollama API endpoint |
| `VISION_MODEL` | `gemma4` | Model name in Ollama (must have vision capability) |

## How It Works

```
┌─────────────┐     ┌───────────────────┐     ┌─────────────┐
│  LLM Client │────▶│  Vision MCP Server │────▶│   Ollama    │
│ (DeepSeek)  │◀────│   (stdio/MCP)      │◀────│  (Gemma 4)  │
└─────────────┘     └───────────────────┘     └─────────────┘
      │                       │
      │ [Image 1] + prompt    │ osascript: clipboard → PNG
      │                       │ base64 → /v1/chat/completions
      ▼                       ▼
  Receives text           Returns vision
  description/OCR         analysis result
```

**Clipboard flow**: User pastes image → LLM calls `process_clipboard_image` → server grabs image from macOS clipboard via `osascript` → encodes to base64 → sends to Ollama → returns text.

**File path flow**: User provides path → LLM calls `describe_image` / `ocr_image` / `ask_image` with path → server reads file → encodes → sends to Ollama → returns text.

## Troubleshooting

| Error | Cause | Fix |
|-------|-------|-----|
| `404 Not Found` | Model doesn't exist in Ollama | `ollama pull gemma4` |
| `Connection refused` | Ollama is not running | `ollama serve` |
| `No image found in clipboard` | Clipboard is empty or not an image | Copy an image to clipboard first |
| Timeout | Model too large for hardware | Switch to a smaller model: `moondream` |

## License

MIT