vision-mcp

A lightweight MCP server that adds vision capabilities to text-only main models. When the main model encounters images (screenshots, UI mockups, flowcharts, error screenshots, etc.) but cannot understand them, it hands the image to an OpenAI-compatible multimodal backend (Qwen-VL, GPT-4o, Gemini, local vLLM, etc.) via MCP tools for analysis and returns the text result.

主模型(纯文本) ──调用 MCP 工具──▶ vision-mcp ──Chat Completions──▶ 多模态模型
   Claude/Codex ◀──────文本结果───────◀──────────────────────────   Qwen-VL / GPT-4o / ...

Quick Start

Recommended: launch directly with the bin/wrapper.sh script bundled in the repo. On first invocation, the wrapper automatically creates a .venv in the directory and installs dependencies (uv preferred; falls back to venv+pip if uv is absent), so no manual installation is needed; afterwards it execs the real server.py. Works on macOS / Linux.

# 方式一（推荐，免手动装依赖）：MCP client 指向 wrapper 即可
claude mcp add vision-mcp -- /绝对/路径/vision-mcp/bin/wrapper.sh

# 或先自测：wrapper 会自举依赖并启动 server
/绝对/路径/vision-mcp/bin/wrapper.sh --check

On Windows, use python -m venv + install.ps1, or just run python server.py.

Related MCP server: vision-mcp

Installation (build from source)

First install the MCP server itself (Python):

# Windows
powershell -ExecutionPolicy Bypass -File .\install.ps1
# macOS / Linux
./install.sh

Then connect it to your client:

Claude Code

claude mcp add vision-mcp -- \
  python /绝对/路径/vision-mcp/server.py

Codex

codex mcp add vision-mcp -- \
  python /绝对/路径/vision-mcp/server.py

After modifying the MCP config, restart the client for changes to take effect.

pi

cp pi-extensions/vision-mcp.ts ~/.pi/agent/extensions/
# 依赖：typebox（必需，工具参数模式定义）；sharp（可选，图片缩放，未装则自动降级为不缩放）
cd ~/.pi/agent/extensions && npm i sharp typebox

After copying, restart pi or use /reload to load automatically; no pi install needed. pi extension supports capability gating: when the main model natively supports images (input includes image), the three vision tools are automatically hidden to avoid redundant delegation; they are only shown for text-only main models.

Configuration

Priority: config.json > process environment variables > .env > defaults. config.json is gitignored and not committed to the repo.

cp config.example.json config.json   # 再编辑 api_key 等字段

{
  "api": "openai-completions",
  "api_key": "sk-your-dashscope-api-key",
  "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
  "model": "qwen-vl-plus",
  "max_tokens": 4096,
  "timeout": 120,
  "max_retries": 2,
  "retry_backoff": 2
}

pi extension additionally reads ~/.pi/vision-mcp/config.json (or a path specified via VISION_CONFIG_PATH), with the same keys as above.

Supported formats: PNG / JPEG / WebP / GIF (BMP is excluded because mainstream vision backends do not support it).

Common backends: DashScope (default) VISION_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1; OpenAI model=gpt-4o; local vLLM base_url=http://localhost:8000/v1.

Checking Configuration

# wrapper 方式（依赖未装则先自举）
./bin/wrapper.sh --check
# 源码自建（依赖手动安装后）
python server.py --check   # 打印生效配置，API key 脱敏

Tools

• vision_analyze — general image understanding. Parameters: prompt (optional), image/image_path/image_url/image_base64 (choose one of four).

• vision_ocr — extracts image text verbatim. Parameters same as analyze (no prompt).

• vision_analyze_batch — batch analysis of multiple images. Parameters: items (required; each item is one of the four image sources, may include prompt), prompt (optional), concurrency (default 3, range 1-8).

Open Source

Released as open source; Fork / Issue / PR are welcome.