llm-vision

Give vision to vision-less LLMs — a local MCP server powered by Alibaba DashScope.

English | 简体中文

Vision-less models (e.g. DeepSeek) can't see images — but they don't have to. llm-vision is a local MCP server that acts as their eyes: hand it a local image path, and it returns a text description generated by Alibaba Cloud's vision models (qwen3-vl-plus / qwen3.5-ocr).

✨ Features

• Two tools, one pipeline — describe_image for general image understanding, extract_text for OCR & document parsing (ID cards, invoices, receipts)

• Bring your own model — model IDs configurable via environment variables, no code changes

• Zero-cost test suite — 68 tests, most run offline against mocked HTTP

• Portable setup — ship .mcp.json with your repo; works anywhere after uv sync

• Model-consumable errors — every tool returns a readable error string, never an exception

🚀 Quick Start

Prerequisites

• Python 3.12+ and uv

• A DashScope API key

1. Install

git clone https://github.com/1710782766/llm_vision.git && cd llm-vision
uv sync
export DASHSCOPE_API_KEY=sk-xxx   # or add to your shell profile

2. Register with Claude Code

The repo ships with a portable .mcp.json — just open Claude Code in the project directory and ask:

"Use describe_image to look at path/to/your/image.jpg and tell me what's in it."

Approve the server connection once, and every future session has vision.

Prefer the CLI? Register manually:

claude mcp add llm-vision --env DASHSCOPE_API_KEY=sk-xxx -- uv run python main.py

Use it in every project — register globally (user scope) so any project directory has vision:

claude mcp add -s user llm-vision --env DASHSCOPE_API_KEY=sk-xxx -- uv run python /absolute/path/to/llm_vision/main.py

⚠️ With global registration, relative image paths resolve against the MCP process's working directory (your current project), not this repo — pass absolute paths to the model.

The project-local .mcp.json (portable, cwd: ".") and global -s user registration serve different setups: the former keeps the server bound to this repo, the latter makes it available everywhere.

🛠 Tools

Supported formats: jpg · jpeg · png · webp · gif · bmp · heic/heif (HEIC/HEIF via macOS sips) — single file < 10 MB.

⚙️ Configuration

🛡️ Reliability & Cost

Designed to "just work" in real use — including screenshot-heavy workflows:

• Auto-compression — images over 1568px (the DashScope recommended edge) are scaled down via the macOS built-in sips (zero runtime dependencies); oversized files are re-encoded (JPEG q85, transparent PNGs preserved). Only oversize images are touched; normal images pass through untouched. Fixes the classic "big screenshot times out" failure. Non-macOS platforms skip preprocessing and rely on timeout + retry instead.

• Retries — transient errors (timeout, network, HTTP 5xx) retry up to LLM_VISION_MAX_RETRIES times with exponential backoff and a shrinking per-attempt budget (total ≤ 2× timeout). Errors are tagged （已重试 N 次） so you know the failure survived retries.

• Result cache — identical image + model + prompt + preprocess settings hits a content-addressed cache (keyed by file SHA-256) at ~/.cache/llm-vision/responses.json; re-viewing the same screenshot costs nothing. Only the model's text answer is stored — never image bytes. The file being replaced (new hash) invalidates the entry automatically. Disable with LLM_VISION_CACHE=0.

  • ⚠️ Note: OCR results (e.g. ID-card numbers extracted via extract_text) are stored in plain text in that cache file for up to 30 days (file permission 0600). For sensitive documents, set LLM_VISION_CACHE=0.

  • Animated GIFs over 1.5 MB are converted to their first frame.

🧠 Model Selection

• qwen3-vl-plus (default vision) — benchmarked as the only hallucination-free option in our model evaluation; qwen3.7-plus is a promising upgrade candidate

• qwen3.5-ocr (default OCR) — cheap, and notably strong at document & card-key-value extraction

• ⚠️ The qwen3-vl-plus-latest alias has been retired (returns 404) — use stable model IDs

🏗 Architecture

main.py (MCP server)
  ├── describe_image(path, prompt?, perspective="normal")   → LLM_VISION_MODEL
  ├── extract_text(path, prompt?)     → LLM_VISION_OCR_MODEL
  └── _analyze_image pipeline
        → image_loader      path/extension/10MB validation, base64 + MIME
        → dashscope_client  httpx → DashScope OpenAI-compatible endpoint

Tools always return a string: the model's answer on success, a readable Chinese error message on failure — never an exception to the client.

🔒 Security & Privacy

• DASHSCOPE_API_KEY lives only in your environment — never in .mcp.json or in git

• When a tool is invoked, the image is sent as base64 to Alibaba DashScope — only hand the model images you're comfortable leaving your machine

🧪 Development

uv run pytest tests/ -q                                   # full suite (22 tests, mostly offline)
uv run python scripts/smoke_test.py [image_path ...]      # real-API smoke test (billed, ~¥0.01/call); pass paths or provide your own under images/
uv run python scripts/compare_models.py qwen3-vl-plus qwen3.7-plus   # model bake-off (billed)

Developer notes for Claude Code: see CLAUDE.md.