Vision MCP Server

MCP server for image processing via Ollama vision models (Gemma 4, Gemma 3, LLaVA...).
Enables LLM clients without vision capability (DeepSeek, Qwen, etc.) to process images by delegating to a local vision model through Ollama.

Features

Related MCP server: llm-vision-mcp

Requirements

• macOS (clipboard tool uses osascript)

• Python 3.12+

• uv — Python package manager

• Ollama — local LLM runtime

Installation

1. Clone the repo

git clone https://github.com/nguyenduc/vision-mcp-server.git
cd vision-mcp-server

2. Install dependencies

uv sync

uv sync creates .venv/ and installs all packages from uv.lock. No need for pip install or uv init.

3. Pull a vision model

ollama pull gemma4

Other compatible vision models: gemma3, llava, llava-llama3, moondream.

4. Make sure Ollama is running

ollama serve

Verify:

curl http://127.0.0.1:11434/api/tags

5. Test the server

uv run server.py

The server runs over stdio — press Ctrl+C to stop.

MCP Client Configuration

OpenCode

Add to .opencode.json (project-level or ~/.opencode.json):

{
  "mcpServers": {
    "vision": {
      "enabled": true,
      "type": "local",
      "command": ["uv", "run", "server.py"],
      "cwd": "/absolute/path/to/vision-mcp-server",
      "env": ["OLLAMA_BASE_URL=http://127.0.0.1:11434", "VISION_MODEL=gemma4"]
    }
  }
}

Note: In OpenCode, command is an array and env is an array of "KEY=VALUE" strings, not an object.

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "vision": {
      "command": "uv",
      "args": ["run", "server.py"],
      "cwd": "/absolute/path/to/vision-mcp-server"
    }
  }
}

Cursor / Windsurf / Cline

{
  "mcpServers": {
    "vision": {
      "command": "uv",
      "args": ["run", "server.py"],
      "cwd": "/absolute/path/to/vision-mcp-server",
      "env": {
        "OLLAMA_BASE_URL": "http://127.0.0.1:11434",
        "VISION_MODEL": "gemma4"
      }
    }
  }
}

Environment Variables

How It Works

┌─────────────┐     ┌───────────────────┐     ┌─────────────┐
│  LLM Client │────▶│  Vision MCP Server │────▶│   Ollama    │
│ (DeepSeek)  │◀────│   (stdio/MCP)      │◀────│  (Gemma 4)  │
└─────────────┘     └───────────────────┘     └─────────────┘
      │                       │
      │ [Image 1] + prompt    │ osascript: clipboard → PNG
      │                       │ base64 → /v1/chat/completions
      ▼                       ▼
  Receives text           Returns vision
  description/OCR         analysis result

Clipboard flow: User pastes image → LLM calls process_clipboard_image → server grabs image from macOS clipboard via osascript → encodes to base64 → sends to Ollama → returns text.

File path flow: User provides path → LLM calls describe_image / ocr_image / ask_image with path → server reads file → encodes → sends to Ollama → returns text.

Troubleshooting

License

MIT