Skip to main content
Glama

🖼️ vision-mcp

Self-hosted multimodal VLM image recognition MCP server

Paste an image in the TUI terminal → the AI client automatically recognizes and returns results · data never leaves your intranet

MCP TypeScript Node Tests Build License: MIT Transport

Claude Code · Codex · OpenCode · any MCP-compatible client


✨ Why use it

Advantage

Description

🔒

Private deployment, data stays on-prem

Connects directly to your self-hosted VLM; images never pass through third-party clouds

🔌

OpenAI-compatible, swappable backend

vLLM / Ollama / GLM-4V / Qwen-VL — pick any, just change the base URL, no code changes

🖼️

Paste-to-recognize in TUI

Paste an image in the terminal, the client automatically calls the tool to recognize it, matching the Zhipu image recognition MCP experience

🧩

Four dedicated tools

General understanding / OCR / diagram understanding / UI-to-code, each with preset system prompts and structured output

📥

Three image input modes

Local path · http(s) URL · data: URI — whatever the client sends, it accepts

🛡️

No error leakage

Error strings contain only static text/status codes; VLM response bodies or stack traces are never leaked to the client

Lightweight single process

stdio; the client spawns the subprocess on demand — no daemon, no server-side state

🔁

Built-in resilience

Auto-retry once on 5xx/timeout, no retry on 4xx, request timeout, image size limit

Full TDD coverage

35 tests + end-to-end round trip (fake VLM + InMemoryTransport)

Related MCP server: readpic MCP Server

📐 Architecture

flowchart LR
    A["🖥️ TUI 客户端<br/>(Claude Code / Codex / OpenCode)"] -- stdio JSON-RPC --> B
    subgraph B["vision-mcp (Node, stdio)"]
        direction TB
        C["tools ×4<br/>analyze_image / extract_text /<br/>understand_diagram / ui_to_code"]
        C --> D["analyze()<br/>共享核心"]
        D --> E["imageSource<br/>路径/URL/data-URI → 归一化"]
        D --> F["vlmClient<br/>OpenAI 兼容 + 重试"]
    end
    F -- HTTPS chat/completions --> G["🧠 自托管 VLM<br/>(qwen-vl / glm-4v / ...)"]
    G -- JSON --> B
    B -- tool result --> A

🛠️ Tools

All share the image_source parameter (local path | http(s) URL | data: URI).

Tool

Tool-specific parameters

Output

analyze_image

prompt (required)

Natural language description / Q&A

extract_text

prompt?, programming_language?

OCR text (code screenshots annotated with language)

understand_diagram

diagram_type? (omit or auto), prompt?

Structured description + mermaid/markdown reproduction

ui_to_code

output_type (code/spec/description), framework? (html/react-tailwind), prompt?

Corresponding code/spec/description

🚀 Quick start

Clone and build

git clone https://github.com/skyone123/vision-mcp.git
cd vision-mcp
npm install
npm run build      # 产出 dist/index.js + dist/index.d.ts
npm test           # 可选:35/35 测试

The client only uses dist/index.jsnote down its absolute path (referred to as $DIST below); you'll need it in the config.

Example: Linux/macOS /home/you/vision-mcp/dist/index.js; Windows D:/git/vision-mcp/dist/index.js.

Environment variables

Variable

Default

Required

Description

VLM_BASE_URL

OpenAI-compatible base, e.g. http://localhost:8000/v1 (include /v1)

VLM_MODEL

qwen-vl-max

Model name

VLM_API_KEY

""

Bearer token; only needed if the backend requires auth — leave empty to omit the Authorization header

VLM_TIMEOUT_MS

60000

Per-request timeout

VLM_MAX_IMAGE_BYTES

10485760

Image size limit 10MB

VLM_MAX_TOKENS

2048

Max tokens in the response

If VLM_BASE_URL is missing, the server exits with an error on startup — no silent failure.

🔧 Configuration

Step 1 · Determine whether the backend needs an API key

curl http://localhost:8000/v1/models
  • 200 + model list → no key needed

  • 401/403key required; retry with the key: curl http://localhost:8000/v1/models -H "Authorization: Bearer your-token"

Pick a vision model from the returned list:

curl -s http://localhost:8000/v1/models | grep '"id"'

Verify it can actually process images (the critical part):

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer 你的token" \
  -d '{
    "model": "qwen-vl-max",
    "messages": [{"role":"user","content":[
      {"type":"text","text":"一句话描述这张图"},
      {"type":"image_url","image_url":{"url":"https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/640px-PNG_transparency_demonstration_1.png"}}
    ]}]
  }'

If it returns normal text → the endpoint works; copy those values into env.

Step 2 · Add it to your client

Replace $DIST below with the absolute path to dist/index.js you noted earlier; use node as the command.

claude mcp add vision-mcp --scope user \
  --env VLM_BASE_URL=http://localhost:8000/v1 \
  --env VLM_MODEL=qwen-vl-max \
  -- node "$DIST"

If a key is required, add another line: --env VLM_API_KEY=your-token.

{
  "command": "node",
  "args": ["/absolute/path/to/vision-mcp/dist/index.js"],
  "env": {
    "VLM_BASE_URL": "http://localhost:8000/v1",
    "VLM_MODEL": "qwen-vl-max"
  }
}

With a key, add "VLM_API_KEY": "your-token" to env.

{
  "mcpServers": {
    "vision-mcp": {
      "command": "node",
      "args": ["/absolute/path/to/vision-mcp/dist/index.js"],
      "env": { "VLM_BASE_URL": "http://localhost:8000/v1", "VLM_MODEL": "qwen-vl-max" }
    }
  }
}
[mcp_servers.vision-mcp]
command = "node"
args = ["/absolute/path/to/vision-mcp/dist/index.js"]
env = { VLM_BASE_URL = "http://localhost:8000/v1", VLM_MODEL = "qwen-vl-max" }
{
  "mcp": {
    "vision-mcp": {
      "type": "local",
      "command": ["node", "/absolute/path/to/vision-mcp/dist/index.js"],
      "environment": {
        "VLM_BASE_URL": "http://localhost:8000/v1",
        "VLM_MODEL": "qwen-vl-max"
      }
    }
  }
}

Field names may vary slightly between OpenCode versions; if the tool doesn't show up, check its official MCP docs.

Step 3 · Verify

claude mcp list          # 应看到 vision-mcp,状态 connected

The MCP server doesn't need to be manually kept running — the client spawns the subprocess on demand. Then paste an image in the conversation and ask "what's in this image" — the client automatically calls analyze_image; or do it explicitly:

Use the analyze_image tool to look at this image:

💻 Development

npm run dev              # tsx 直接跑源码(开发期)
npm run build            # tsup 打包 dist/index.js
npm test                 # vitest,35/35
npx tsc --noEmit         # 类型检查

Source structure:

src/
  config.ts          # env → VlmConfig
  imageSource.ts     # loadImage: 路径/URL/data-URI 归一化
  vlmClient.ts       # complete: 调 OpenAI 兼容端点 + 重试/超时
  analyze.ts         # 共享核心: loadImage + complete
  server.ts          # McpServer 注册 + stdio + main
  index.ts           # #!/usr/bin/env node 入口
  tools/
    analyzeImage.ts
    extractText.ts
    understandDiagram.ts
    uiToCode.ts

Each file has a single responsibility and can be tested independently; the four tools are thin wrappers around analyze(), each baking in its own system prompt.

🗺️ Roadmap (optional extensions)

Current scope: stdio only · single backend · single image · no persistence. The following are on-demand extensions:

Candidate

Value

Recommendation

Streaming output

ui_to_code output can be long; streaming lets you see it as it's generated

👍 Worth doing, better UX

Image preprocessing

Scale/compress by long edge before sending — saves tokens, reduces timeouts

👍 Worth doing, lower cost

Structured output

extract_text/understand_diagram return JSON

🤔 Depends on the use case

HTTP/SSE transport

Shared by multiple clients, remote deployment

🤔 stdio is enough for now; on demand

Multi-backend routing

Route different tasks to different VLMs

❌ YAGNI

Video/multi-image batch processing

❌ Beyond current scope

Server-side caching

Cache recognition of identical images

❌ YAGNI

📄 License

MIT © 2026 luyuxin


A
license - permissive license
Not graded
quality - not tested
A
maintenance

Maintenance

Maintainers
Response time
Release cycle
1Releases (12mo)
Commit activity

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Servers

View all related MCP servers

Related MCP Connectors

  • OCR, transcription, file extraction, and image generation for AI agents via MCP.

  • Generate images with any major model — one API key, one prepaid balance, one MCP.

  • Self-hosted MCP gateway: turn any API, database or MCP server into AI connectors — no code.

View all MCP Connectors

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/skyone123/vision-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server