modlens-mcp

MCP server that gives text-only models (e.g. DeepSeek) structured JSON vision evidence (summary / OCR / layout / semantics / uncertainty) by calling vision APIs directly — no CLI dependencies. The output contract follows the ModLens spec, so evidence files stay structured and machine-checkable.

Designed for Reasonix (.mcp.json with Claude Code-compatible mcpServers), works with any MCP client.

Features

• 4 tools — analyze_image, ocr_image, compare_images, find_image

• Direct API calls: OpenAI-compatible endpoints (openai / gemini-api) and Anthropic native (anthropic) — configured via one JSON file, no CLI installs, no shell spawning

• Structured evidence, never free text: the ModLens JSON contract (summary / ocr / layout / semantics / uncertainty) is requested via a strict system prompt and validated on parse

• Fits Reasonix's 32KB tool-result cap: every tool result is a slim projection (summary / uncertainty / evidence_path); the full evidence JSON is persisted to .reasonix/vision-evidence/<sha256>.json and read on demand via read_file

• Stateless & zero-pollution: each call is an independent HTTP request; image bytes never enter any model history

• Registry: analyze_image registers image metadata (.reasonix/vision-index.jsonl) so find_image can locate pasted images even after the conversation window lost the reference

• Workspace root resolution: on every call the server asks the MCP client for the current project root (roots/list, ≤1s, never cached across calls) and probes candidate roots in order — project root → configured workspaceRoot → global workspace ~/.reasonix → home dir → cwd — picking the first that exists. A single long-lived server resolves pasted attachments correctly in every project (Reasonix stores pasted files under the project's .reasonix/attachments/).

• Delivery-first friendly: all four tools declare readOnlyHint, so Reasonix hosts treat them as non-destructive instead of blocking them behind acceptance criteria. Side effects stay inside the server's own state (~/.reasonix/vision-evidence/, vision-index.jsonl); the only external effect is the paid vision-API call you configured.

Related MCP server: omni-vision-pro

Install

npm install          # inside this project
npm run build        # tsc → dist/

Configure

Precedence: Reasonix MCP panel env wins — set VISION_API_KEY (plus VISION_PROVIDER / VISION_BASE_URL / VISION_MODEL) in the MCP server env (panel or .mcp.json) and it takes effect after restart. Without a key in env, the server reads ~/.reasonix/vision-config.json (copy from config.example.json; VISION_CONFIG_FILE overrides the path), then falls back to the remaining VISION_* env vars.

{
  "provider": "openai",
  "baseUrl": "https://open.bigmodel.cn/api/paas/v4",
  "apiKey": "your-real-key",
  "model": "glm-4.6v",
  "workspaceRoot": "D:/你的项目目录"
}

• provider: openai (OpenAI-compatible endpoints: Zhipu GLM, Aliyun Qwen-VL, OpenAI…), gemini-api (its OpenAI-compatible endpoint), or anthropic (native Messages API; baseUrl defaults to https://api.anthropic.com)

• Missing / placeholder / non-ASCII keys are rejected with a clear Chinese error before any HTTP call — you can never hit a cryptic provider error

.mcp.json (project root) only needs the server itself:

{
  "mcpServers": {
    "vision": {
      "command": "node",
      "args": ["<你的用户目录>/.reasonix/modlens-mcp/dist/index.js"]
    }
  }
}

No VISION_WORKSPACE_ROOT needed: relative paths resolve via candidate roots (global workspace ~/.reasonix first), so the same server works in any project.

Environment variables (all optional)

Tool contract (summary)

analyze_image(image, prompt?, provider?, model?) → { image, tool, provider, model, summary, uncertainty, analysis, evidence_path } (slim, <32KB); full evidence (ocr.full_text, layout, semantics) at evidence_path.

Every call makes two vision requests: ① OCR/evidence (ModLens v2/v3 output contract, slimmed for output stability — summary / ocr.full_text / ocr.lines[].text / layout.regions[] (5-type enum: title|paragraph|list|table|other) / semantics{scene,intent,entities,relations} / visual (optional) / uncertainty; no bbox, no per-line language, no per-entity evidence, and lines/regions must not repeat full_text — v2 dropped fabricated coordinates) → evidence file, ② a free-form analysis answering your prompt verbatim (no JSON constraints; a Chinese default question is used when no prompt is given). The analysis lands in the analysis field of both the slim result and the evidence file (additive, backward compatible — legacy v1 shapes with bbox/blocks still parse). If the analysis request fails, the OCR evidence still succeeds (degraded, no analysis).

Tool split (v4): analyze_image runs semantic structure (summary/layout.regions/semantics/uncertainty, ModLens v2/v3 contract minus ocr) in parallel with a free-form analysis of your prompt and merges them into one evidence file. ocr_image is the pure-OCR service (single request, preview + full_text_chars + evidence_path). Both tools key evidence by the IMAGE sha256, so calling them in parallel for the same image merges into ONE json file (result keeps ocr + summary/layout/ semantics + analysis); a write queue serializes concurrent merges. A global API gate (max 5 concurrent vision requests) plus a one-shot retry (3s backoff on 5xx/429/timeout) for analysis keep multi-image scenarios stable. if an endpoint (or a local proxy that forces streaming) still answers with text/event-stream, the SSE payload is parsed transparently — full-completion blocks and delta chunk streams are both supported, [DONE] is ignored. Reasoning models whose content is null fall back to reasoning_content only when it contains our JSON contract; otherwise a clear error is returned.

Output length is not capped: OpenAI-compatible providers (Zhipu, Gemini, OpenAI, ...) receive no max_tokens field and use the model's own maximum output — whatever the model returns is persisted to the evidence file and the slim projection stays <32KB. Anthropic requires max_tokens, so it gets a 16384 fallback. Full-length model output can therefore never be cut off mid-JSON.

ocr_image(image, format?) → { image, format, evidence_path, preview, full_text_chars }.

compare_images(images[2-4], prompt?) → per-image slim blocks with independent image + evidence_path.

find_image(filename?, sha256Prefix?, limit?) → registry matches (metadata only, never image bytes).

Test

npm test    # 95 tests: config + direct (mock HTTP) + tools + real MCP protocol e2e

Security

• No shell spawning; HTTP requests use argument-safe fetch with only the configured endpoint

• Errors are sanitized (keys / Bearer / Basic redacted) before reaching the model context; the config key is never echoed back

• Evidence files and the registry are written with 0600 / 0700 permissions

• The key lives either in ~/.reasonix/vision-config.json (0600, not in version control) or in the Reasonix MCP panel env (VISION_API_KEY, persisted plaintext in the Reasonix config.toml — /mcp shows it redacted; keep that config out of shared screens and backups). Either way: rotate the key if it ever appears in logs or chats