# vision-mcp

An MCP server that gives **conversational image understanding** to text-only models (DeepSeek, Claude Code, etc.).

The main model hands image paths and a question to a vision model, which returns a text description that the main model can reason over. A session mechanism enables "blind men and the elephant" style follow-ups: you can dig deeper into the same batch of images across multiple turns, and the vision model re-sees the full images and the conversation history on every turn.

## Features

- **Multi-turn conversational follow-up**: follow-ups within a session automatically carry history context, supporting referential questions ("What does that sign say?")
- **OpenAI-compatible vision API**: any compatible endpoint works (default SiliconFlow; Qwen3.5-35B-A3B verified to accept images)
- **Path deduplication**: paths passed on session reuse are compared against existing ones; only new images are added
- **URL support**: pass http/https image URLs directly; they are forwarded to the vision API as-is (no local download)
- **Image integrity validation**: checks extension vs. actual format consistency; supports png/jpg/jpeg/webp/gif/bmp/tif/tiff
- **Concurrency safe**: operations on the same session are serialized; atomic writes; deletion is mutually exclusive with in-flight requests, so a deleted session can never be resurrected by a stale save
- **Auto-expiry**: sessions idle for 24 hours are cleaned up (configurable)
- **Passthrough by default**: images are sent as-is — no compression, no scaling, no re-encoding — unless compression is enabled (see `max_image_mb`)
- **Staged compression (optional)**: when enabled, images over the threshold are compressed in stages, format-preserving where possible (JPEG/WebP lower quality first, then downscale; PNG keeps transparency by downscaling before falling back to JPEG)

## Installation

```bash
git clone https://github.com/whyneedai/vision-mcp.git
cd vision-mcp
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
```

## Configuration

The config file lives at `~/.config/vision-mcp/config.json`:

```json
{
  "vision_model": {
    "base_url": "https://api.siliconflow.cn/v1",
    "api_key": "{env:SILICONFLOW_API_KEY}",
    "model": "Qwen/Qwen3.5-35B-A3B",
    "enable_thinking": false,
    "max_tokens": 131072,
    "temperature": 0.1
  },
  "max_history_rounds": 4,
  "sessions_dir": "~/.local/share/vision-mcp/sessions",
  "session_ttl_hours": 24,
  "system_prompt": "optional, overrides the built-in vision system prompt"
}
```

| Field | Description |
| --- | --- |
| `max_history_rounds` | Number of recent Q&A rounds (1 round = one question + one answer) sent to the vision model as context. History beyond the window is kept on disk but not sent. Default `4`; `0` sends no history. Capped by the provider's message limit (10) |
| `vision_model.base_url` / `api_key` / `model` | OpenAI-compatible endpoint; `{env:XXX}` references an environment variable |
| `vision_model.enable_thinking` | Disable thinking mode (otherwise the API returns an empty `content`) |
| `vision_model.max_image_mb` | Compression threshold in MB: images at or above this size are auto-compressed below it (staged, format-preserving). Unset / empty / `0` disables compression (default) |
| `sessions_dir` | Session storage directory (default `~/.local/share/vision-mcp/sessions`) |
| `session_ttl_hours` | Session idle-expiry in hours (default 24) |
| `system_prompt` | Vision system prompt (default: strictly follow the question, no hallucination) |

## Connecting to opencode

Add to the `mcp` section of your opencode config:

```json
{
  "mcp": {
    "vision": {
      "type": "local",
      "command": ["/path/to/vision-mcp/.venv/bin/python", "/path/to/vision-mcp/server.py"],
      "environment": {
        "SILICONFLOW_API_KEY": "{env:SILICONFLOW_API_KEY}"
      },
      "enabled": true,
      "timeout": 300000
    }
  }
}
```

## Tools

### `ask_image`

Ask a question about one or more images, with multi-turn follow-up support.

| Parameter | Required | Description |
| --- | --- | --- |
| `question` | yes | The question |
| `session_id` | no | Existing session ID; omit to create a new session (`image_path` is then required) |
| `image_path` | no | List of image paths **or http/https image URLs**; may be omitted on session reuse (existing images are kept), new entries are deduplicated and appended. URLs are forwarded as-is to the vision API |

Returns `{session_id, answer, image_paths}`. Relative paths resolve against the opencode working directory.

### `end_session`

Delete a session and all of its related files. Original images are never deleted.

## Architecture

```
server.py   MCP entry point: component wiring + tool registration
config.py   Config loading ({env:XXX} resolution)
sessions.py Session storage: atomic writes, per-session locks, TTL cleanup
images.py   Path/content validation, staged compression, data URL encoding
vision.py   Vision client: OpenAI-compatible API
```

## Tests

```bash
.venv/bin/python test/test_mcp_proto.py   # MCP handshake and tool registration
.venv/bin/python test/test_e2e.py         # end-to-end (requires a real API key)
```

## License

MIT