# mcp-vision-server

English | [中文](README.zh.md)

MCP server that exposes an `analyze_image` tool: it forwards an image to a configurable OpenAI-compatible vision model and returns the model's text answer. Any MCP client — DSH through `@deepseek-ai/dsh-mcp-client`, Claude Desktop, or others — can call it.

## Usage

Start the server over stdio:

```sh
export VISION_API_KEY=...
export VISION_MODEL=gpt-4o
npx -y mcp-vision-server
```

Pass `--help` to print the environment-variable reference. To run from a checkout instead of npm: `pnpm install && pnpm build && node lib/bin.js`.

The bin also reads a `.env` file placed next to it (copy `.env.example` to `.env` and fill it in); variables already present in the environment always win over `.env` values.

### Connect from DSH

Add one `@deepseek-ai/dsh-mcp-client` instance to `cordis.yml`:

```yaml
- id: mcp-vision
  name: '@deepseek-ai/dsh-mcp-client'
  config:
    serverName: vision
    transport: stdio
    command: npx
    args: ['-y', 'mcp-vision-server']
    env:
      VISION_API_BASE_URL: https://open.bigmodel.cn/api/paas/v4
      VISION_API_KEY: !!js process.env.VISION_API_KEY
      VISION_MODEL: glm-4v-plus
```

The model sees the tool as `mcp__vision__analyze_image`.

### Connect from Claude Desktop

`claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "vision": {
      "command": "npx",
      "args": ["-y", "mcp-vision-server"],
      "env": {
        "VISION_API_KEY": "...",
        "VISION_MODEL": "gpt-4o"
      }
    }
  }
}
```

## Tool: analyze_image

`analyze_image(image, question?)`

| Argument | Required | Description |
|---|---|---|
| `image` | yes | Image source: `http(s)://` URL, local absolute path, or `data:<mime>;base64,<data>` URL |
| `question` | no | Question to answer about the image; defaults to `VISION_DEFAULT_PROMPT` |

The tool returns the vision model's text answer. Failures return an `isError` result with a readable Chinese message instead of throwing. Image bytes travel only to the configured vision API and never enter the calling client's model context.

## Configuration

| Env var | Required | Default | Description |
|---|---|---|---|
| `VISION_API_BASE_URL` | no | `https://api.openai.com/v1` | OpenAI-compatible API root URL, without a trailing slash |
| `VISION_API_KEY` | yes | — | API key; startup fails loud when missing |
| `VISION_MODEL` | yes | — | Vision model name, e.g. `gpt-4o`, `glm-4v-plus`, `qwen-vl-max` |
| `VISION_DEFAULT_PROMPT` | no | `请详细描述这张图片的内容。` | Prompt used when the caller passes no question |
| `VISION_MAX_IMAGE_BYTES` | no | `20971520` | Upper bound on accepted image bytes |
| `VISION_TIMEOUT_MS` | no | `120000` | Vision API call timeout in milliseconds |

## Model Experience

### analyze_image tool (served through the MCP client bridge)

#### What the model sees

The tool is served to MCP clients with the pinned name `analyze_image`, description, and input schema. The description is verbatim:

##### Verbatim tool description

```markdown
分析一张图片：传入图片 URL、本地绝对路径或 data URL（base64），可附带问题；返回视觉模型的文字回答。
```

##### Input schema field descriptions

```markdown
image: 图片来源：http(s) URL、本地绝对路径或 data:image/...;base64,...
question: 可选问题；缺省时默认要求描述图片
```

#### Token effect

Fixed schema cost while a client keeps the tool registered, plus the tool result text in each conversation that calls it. The image bytes themselves never enter the client's model context.

#### KV Cache effect

Each `analyze_image` call issues an independent model request to the configured vision API, separate from the client's own request prefix; the server's behavior never invalidates client-side reuse. The bridged result text grows the client conversation append-only.

## Known Limitations and Deferred Work

- **stdio only** — no Streamable HTTP transport; the server is not reachable over the network.
- **Single image per call** — one `image` argument only; multi-image analysis is deferred.
- **Tools only** — the server exposes no MCP resources or prompts.
- **No image transformation** — bytes are passed through as received; large or exotic formats depend on the vision provider's tolerance.
- **No URL allowlist** — the server downloads whatever URL the caller passes, with no SSRF defense; keep it away from untrusted callers.

## Development

```sh
pnpm install
pnpm test        # unit tests, 100% per-file coverage gate
pnpm test:e2e    # keyless stdio e2e against a stub vision API
pnpm build       # emit lib/
```

The e2e suite spawns the real bin over stdio and serves a stub OpenAI-compatible API over HTTP; no API key is needed.

## Vision Bridge (paste images into text-only models)

[`vision-bridge.mjs`](vision-bridge.mjs) is an OpenAI-compatible strip proxy (the same pattern as [codex-vision-bridge](https://github.com/tkr520521/codex-vision-bridge)) that gives text-only models image input: requests carrying image parts are answered by the vision MCP server, text-only requests pass through unchanged to the real text-only upstream.

```sh
# 1. Fill in the text-upstream key (only needed for text passthrough)
copy vision-bridge.env.example vision-bridge.env
# 2. Start the bridge
start-vision-bridge.cmd        # or: node vision-bridge.mjs
# 3. Point a provider at http://127.0.0.1:8787/v1 and declare the model
#    with input: [text, image] (e.g. a pi-ai provider entry in settings.yaml)
```

Fail-open: when the vision service fails, the request falls back to passthrough instead of breaking.