Vision MCP

Enable LLMs without vision capabilities (such as GLM) to process images in Claude Code / Codex.

How It Works

When the main model encounters an image, it calls the describe_image tool provided by this MCP:

1. The tool reads the local image file

2. It calls the vision model you configured (Qwen-VL / GPT-4o / GLM-4V / Doubao, or any OpenAI-compatible endpoint)

3. It returns a text description to the main model

4. The main model continues reasoning based on the text

The main model never touches the image binary, so pure-text models can use it too.

The project has zero third-party dependencies (pure Python standard library), requires only Python 3.8+, and needs no pip installation.

Related MCP server: Image-Vision MCP Server

One-Click Installation

git clone https://github.com/Ruiba0/Vision-MCP.git
cd vision-mcp
python3 install.py

On Windows there is no python3 command; use python install.py instead. The same applies to all python3 references below.

The installation script interactively handles:

• Asking you for the vision model's base URL / model name / API key (with common vendor references)

• Registering the MCP with Claude Code (claude mcp add)

• Registering the MCP with Codex (writing to ~/.codex/config.toml)

• Adding image processing rules to ~/.claude/CLAUDE.md and ~/.codex/AGENTS.md

The script is idempotent and can be re-run.

Manual Installation

If you prefer not to use the installation script:

1. Configure the vision model

   Copy config.example.json to config.json and fill in your vision model configuration:

   {
     "vision_api_base": "https://dashscope.aliyuncs.com/compatible-mode/v1",
     "vision_model": "qwen-vl-max",
     "vision_api_key": "sk-你的key",
     "max_image_bytes": 10485760
   }

2. Register with Claude Code

   claude mcp add --scope user vision -- python3 /绝对路径/server.py

   Add --scope user to make it available globally (without it, it only takes effect in the current directory).

3. Register with Codex

   Edit ~/.codex/config.toml and add:

   [mcp_servers.vision]
   command = "python3"
   args = ["/绝对路径/server.py"]

4. Add image processing rules

   Add the following to ~/.claude/CLAUDE.md and ~/.codex/AGENTS.md (so the main model proactively calls the tool when it encounters images):

   ## 图片处理
   
   当用户提到图片文件路径（.png/.jpg/.jpeg/.gif/.webp/.bmp），或要求查看/分析/识别某个图片文件时：
   
   - 先判断当前主模型自身是否具备视觉能力
   - 具备视觉（如 Claude Sonnet/Opus、GPT-4o、Qwen-VL 等多模态模型）→ 直接读取图片并分析
   - 不具备视觉（如 GLM 等纯文本模型）→ 调用 vision MCP 的 describe_image 工具，传入图片路径和问题
   - 不确定自身是否支持视觉时，默认调用 describe_image 工具作为兜底
   - 用户明确要求"用 MCP 看"或"调视觉模型"时，无论主模型是否支持视觉，都调用 describe_image 工具

Supported Vision Models

Any vision model compatible with the OpenAI protocol can be used:

To use Anthropic's native protocol (such as Claude), you will need to modify vision_client.py yourself; currently only the OpenAI-compatible protocol is supported.

Usage

After installation, restart Claude Code / Codex and in a conversation:

• "Take a look at this image at ~/Desktop/diagram.png"

• "Analyze the error in ~/Screenshots/error.jpg"

• "Extract all the text in ~/Documents/notes/page1.jpeg"

(On Windows, paths look like D:/temp/diagram.png; write them according to your system)

The main model will automatically call the describe_image tool, which returns the vision model's text description, and the main model continues answering based on that description.

Tool Parameters

describe_image(path: str, question: str = "Describe the content of this image in detail") -> str

• path: absolute path to the image file; supports png/jpg/jpeg/gif/webp/bmp

• question: the question to ask the vision model; can be adjusted per scenario ("extract text", "describe layout", "analyze chart", etc.)

Project Structure

vision-mcp/
├── server.py              # MCP 服务（内置 stdio JSON-RPC 实现，无 SDK 依赖）
├── vision_client.py       # 视觉模型调用（OpenAI 兼容协议，urllib 实现）
├── config.example.json    # 配置模板（提交到 git）
├── config.json            # 你的实际配置（gitignored）
├── install.py             # 一键安装脚本
└── README.md

FAQ

Q: The call reports vision model connection failed Check whether the vision_api_base path is correct. The tool appends /chat/completions after the base URL, so the base URL should not include the /chat/completions suffix.

Q: The call reports HTTP 401 The API key is invalid or you lack permission for that model. Check whether vision_api_key and vision_model match the corresponding platform.

Q: The main model does not proactively call the tool Confirm that the image processing rules in CLAUDE.md / AGENTS.md have been added. The keywords in the rules (image, view, analyze) guide the main model to call describe_image.

Q: I want to switch vision models Edit the three fields in config.json; no code changes are needed. Or re-run python3 install.py to reconfigure.

Dependencies

• Python 3.8+ (pure standard library, no pip packages required)

License

MIT