<div align="center">

# llm-vision

**Give vision to vision-less LLMs — a local MCP server powered by Alibaba DashScope.**

[![Python](https://img.shields.io/badge/Python-3.12-blue.svg)](https://www.python.org/downloads/)
[![MCP](https://img.shields.io/badge/MCP-Server-purple.svg)](https://modelcontextprotocol.io)
[![DashScope](https://img.shields.io/badge/DashScope-Alibaba-orange.svg)](https://dashscope.aliyun.com/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

**English | [简体中文](README.zh-CN.md)**

</div>

Vision-less models (e.g. DeepSeek) can't see images — but they don't have to. `llm-vision` is a local [MCP](https://modelcontextprotocol.io) server that acts as their eyes: hand it a local image path, and it returns a text description generated by Alibaba Cloud's vision models (`qwen3-vl-plus` / `qwen3.5-ocr`).

---

## ✨ Features

- **Two tools, one pipeline** — `describe_image` for general image understanding, `extract_text` for OCR & document parsing (ID cards, invoices, receipts)
- **Bring your own model** — model IDs configurable via environment variables, no code changes
- **Zero-cost test suite** — 68 tests, most run offline against mocked HTTP
- **Portable setup** — ship `.mcp.json` with your repo; works anywhere after `uv sync`
- **Model-consumable errors** — every tool returns a readable error string, never an exception

## 🚀 Quick Start

### Prerequisites

- Python 3.12+ and [uv](https://docs.astral.sh/uv/)
- A [DashScope](https://dashscope.aliyun.com/) API key

### 1. Install

```bash
git clone https://github.com/1710782766/llm_vision.git && cd llm-vision
uv sync
export DASHSCOPE_API_KEY=sk-xxx   # or add to your shell profile
```

### 2. Register with Claude Code

The repo ships with a portable `.mcp.json` — just open Claude Code in the project directory and ask:

> *"Use `describe_image` to look at `path/to/your/image.jpg` and tell me what's in it."*

Approve the server connection once, and every future session has vision.

Prefer the CLI? Register manually:

```bash
claude mcp add llm-vision --env DASHSCOPE_API_KEY=sk-xxx -- uv run python main.py
```

**Use it in every project** — register globally (user scope) so any project directory has vision:

```bash
claude mcp add -s user llm-vision --env DASHSCOPE_API_KEY=sk-xxx -- uv run python /absolute/path/to/llm_vision/main.py
```

> ⚠️ With global registration, relative image paths resolve against the MCP process's working directory (your current project), not this repo — **pass absolute paths to the model**.

The project-local `.mcp.json` (portable, `cwd: "."`) and global `-s user` registration serve different setups: the former keeps the server bound to this repo, the latter makes it available everywhere.

## 🛠 Tools

| Tool | Arguments | Description |
|---|---|---|
| `describe_image` | `image_path` (required) · `prompt` (optional) · `perspective` (optional) | View an image; `perspective=normal` (default) natural description, `perspective=critical` inspecting — proactively reports text misalignment/overlap/missing elements, for screenshot bug-hunting |
| `extract_text` | `image_path` (required) · `prompt` (optional) | OCR & text localization — documents, ID cards, invoices; ask for structured output (e.g. *"extract the name and ID number as JSON"*) |

Supported formats: `jpg · jpeg · png · webp · gif · bmp · heic/heif` (HEIC/HEIF via macOS `sips`) — single file < 10 MB.

## ⚙️ Configuration

| Variable | Required | Default | Description |
|---|---|---|---|
| `DASHSCOPE_API_KEY` | ✅ | — | DashScope API key (`sk-` prefix) |
| `LLM_VISION_MODEL` | — | `qwen3-vl-plus` | Vision model used by `describe_image` |
| `LLM_VISION_OCR_MODEL` | — | `qwen3.5-ocr` | OCR model used by `extract_text` |
| `LLM_VISION_TIMEOUT` | — | `120` | Per-attempt timeout (seconds) |
| `LLM_VISION_MAX_RETRIES` | — | `2` | Retries for transient errors (timeouts, network, HTTP 5xx); `0` disables |
| `LLM_VISION_DESCRIBE_PROMPT` | — | *(built-in)* | Override the default prompt for `perspective=critical` (normal uses a built-in natural-description prompt) |
| `LLM_VISION_OCR_PROMPT` | — | *(built-in)* | Override the default OCR prompt |
| `LLM_VISION_CACHE` | — | `1` | Result cache on/off (`0/false/off/no` disables) |
| `LLM_VISION_CACHE_DIR` | — | `~/.cache/llm-vision` | Cache location (or `$XDG_CACHE_HOME/llm-vision`) |
| `LLM_VISION_MAX_EDGE` | — | `1568` | Max image edge (px) before auto-scaling; `0` disables scaling |
| `LLM_VISION_COMPRESS` | — | `1` | Auto-preprocess oversize images on/off |

## 🛡️ Reliability & Cost

Designed to "just work" in real use — including screenshot-heavy workflows:

- **Auto-compression** — images over 1568px (the DashScope recommended edge) are scaled down via the macOS built-in `sips` (zero runtime dependencies); oversized files are re-encoded (JPEG q85, transparent PNGs preserved). Only oversize images are touched; normal images pass through untouched. Fixes the classic "big screenshot times out" failure. Non-macOS platforms skip preprocessing and rely on timeout + retry instead.
- **Retries** — transient errors (timeout, network, HTTP 5xx) retry up to `LLM_VISION_MAX_RETRIES` times with exponential backoff and a shrinking per-attempt budget (total ≤ 2× timeout). Errors are tagged `（已重试 N 次）` so you know the failure survived retries.
- **Result cache** — identical image + model + prompt + preprocess settings hits a content-addressed cache (keyed by file SHA-256) at `~/.cache/llm-vision/responses.json`; re-viewing the same screenshot costs nothing. Only the model's text answer is stored — never image bytes. The file being replaced (new hash) invalidates the entry automatically. Disable with `LLM_VISION_CACHE=0`.
  - ⚠️ Note: OCR results (e.g. ID-card numbers extracted via `extract_text`) are stored **in plain text** in that cache file for up to 30 days (file permission `0600`). For sensitive documents, set `LLM_VISION_CACHE=0`.
  - Animated GIFs over 1.5 MB are converted to their first frame.

## 🧠 Model Selection

- **`qwen3-vl-plus`** (default vision) — benchmarked as the only hallucination-free option in our [model evaluation](docs/model-eval-2026-08-05.md); `qwen3.7-plus` is a promising upgrade candidate
- **`qwen3.5-ocr`** (default OCR) — cheap, and notably strong at document & card-key-value extraction
- ⚠️ The `qwen3-vl-plus-latest` alias has been retired (returns `404`) — use stable model IDs

## 🏗 Architecture

```
main.py (MCP server)
  ├── describe_image(path, prompt?, perspective="normal")   → LLM_VISION_MODEL
  ├── extract_text(path, prompt?)     → LLM_VISION_OCR_MODEL
  └── _analyze_image pipeline
        → image_loader      path/extension/10MB validation, base64 + MIME
        → dashscope_client  httpx → DashScope OpenAI-compatible endpoint
```

Tools always return a **string**: the model's answer on success, a readable Chinese error message on failure — never an exception to the client.

## 🔒 Security & Privacy

- `DASHSCOPE_API_KEY` lives only in your environment — never in `.mcp.json` or in git
- When a tool is invoked, the image is sent as base64 to Alibaba DashScope — only hand the model images you're comfortable leaving your machine

## 🧪 Development

```bash
uv run pytest tests/ -q                                   # full suite (22 tests, mostly offline)
uv run python scripts/smoke_test.py [image_path ...]      # real-API smoke test (billed, ~¥0.01/call); pass paths or provide your own under images/
uv run python scripts/compare_models.py qwen3-vl-plus qwen3.7-plus   # model bake-off (billed)
```

Developer notes for Claude Code: see [CLAUDE.md](CLAUDE.md).