# modlens-mcp

MCP server that gives text-only models (e.g. DeepSeek) structured **JSON vision
evidence** (summary / OCR / layout / semantics / uncertainty) by calling vision
APIs **directly** — no CLI dependencies. The output contract follows the
ModLens spec, so evidence files stay structured and machine-checkable.

Designed for Reasonix (`.mcp.json` with Claude Code-compatible `mcpServers`),
works with any MCP client.

## Features

- **4 tools** — `analyze_image`, `ocr_image`, `compare_images`, `find_image`
- **Direct API calls**: OpenAI-compatible endpoints (`openai` / `gemini-api`)
  and Anthropic native (`anthropic`) — configured via one JSON file, no CLI
  installs, no shell spawning
- **Structured evidence, never free text**: the ModLens JSON contract
  (`summary` / `ocr` / `layout` / `semantics` / `uncertainty`) is requested via
  a strict system prompt and validated on parse
- **Fits Reasonix's 32KB tool-result cap**: every tool result is a slim
  projection (`summary` / `uncertainty` / `evidence_path`); the full evidence
  JSON is persisted to `.reasonix/vision-evidence/<sha256>.json` and read on
  demand via `read_file`
- **Stateless & zero-pollution**: each call is an independent HTTP request;
  image bytes never enter any model history
- **Registry**: `analyze_image` registers image metadata
  (`.reasonix/vision-index.jsonl`) so `find_image` can locate pasted images
  even after the conversation window lost the reference
- **Workspace root resolution**: on every call the server asks the MCP client
  for the **current project root** (`roots/list`, ≤1s, never cached across
  calls) and probes candidate roots in order — project root → configured
  `workspaceRoot` → global workspace `~/.reasonix` → home dir → cwd — picking
  the first that exists. A single long-lived server resolves pasted
  attachments correctly in every project (Reasonix stores pasted files under
  the project's `.reasonix/attachments/`).
- **Delivery-first friendly**: all four tools declare `readOnlyHint`, so
  Reasonix hosts treat them as non-destructive instead of blocking them behind
  acceptance criteria. Side effects stay inside the server's own state
  (`~/.reasonix/vision-evidence/`, `vision-index.jsonl`); the only external
  effect is the paid vision-API call you configured.

## Install

```bash
npm install          # inside this project
npm run build        # tsc → dist/
```

## Configure

Precedence: **Reasonix MCP panel env wins** — set `VISION_API_KEY` (plus
`VISION_PROVIDER` / `VISION_BASE_URL` / `VISION_MODEL`) in the MCP server env
(panel or `.mcp.json`) and it takes effect after restart. Without a key in env,
the server reads `~/.reasonix/vision-config.json` (copy from
`config.example.json`; `VISION_CONFIG_FILE` overrides the path), then falls
back to the remaining `VISION_*` env vars.

```json
{
  "provider": "openai",
  "baseUrl": "https://open.bigmodel.cn/api/paas/v4",
  "apiKey": "your-real-key",
  "model": "glm-4.6v",
  "workspaceRoot": "D:/你的项目目录"
}
```

- `provider`: `openai` (OpenAI-compatible endpoints: Zhipu GLM, Aliyun
  Qwen-VL, OpenAI…), `gemini-api` (its OpenAI-compatible endpoint), or
  `anthropic` (native Messages API; `baseUrl` defaults to
  `https://api.anthropic.com`)
- Missing / placeholder / non-ASCII keys are rejected with a clear Chinese
  error before any HTTP call — you can never hit a cryptic provider error

`.mcp.json` (project root) only needs the server itself:

```json
{
  "mcpServers": {
    "vision": {
      "command": "node",
      "args": ["<你的用户目录>/.reasonix/modlens-mcp/dist/index.js"]
    }
  }
}
```

No `VISION_WORKSPACE_ROOT` needed: relative paths resolve via candidate roots
(global workspace `~/.reasonix` first), so the same server works in any project.

## Environment variables (all optional)

| Variable | Default | Purpose |
|---|---|---|
| `VISION_CONFIG_FILE` | `~/.reasonix/vision-config.json` | path to the config JSON |
| `VISION_WORKSPACE_ROOT` | config `workspaceRoot` → roots/list | base for relative `@.reasonix/...` paths |
| `VISION_INDEX_FILE` | `<workspace>/.reasonix/vision-index.jsonl` | registry file |
| `VISION_EVIDENCE_DIR` | `<workspace>/.reasonix/vision-evidence` | full evidence JSON output |
| `MODLENS_TIMEOUT_MS` | 180000 | per-call timeout, clamped to [1000, 600000] |
| `VISION_PROVIDER` | `openai` | fallback provider: `openai` \| `gemini-api` \| `anthropic` |
| `VISION_BASE_URL` | provider default | fallback endpoint |
| `VISION_MODEL` | — | fallback model name |
| `VISION_API_KEY` | — | provider key; takes precedence over the config file when set (panel/env wins) |

## Tool contract (summary)

`analyze_image(image, prompt?, provider?, model?)` →
`{ image, tool, provider, model, summary, uncertainty, analysis, evidence_path }`
(slim, <32KB); full evidence (ocr.full_text, layout, semantics) at
`evidence_path`.

Every call makes **two vision requests**: ① OCR/evidence (ModLens **v2/v3
output contract, slimmed for output stability** — `summary` /
`ocr.full_text` / `ocr.lines[].text` / `layout.regions[]` (5-type enum:
title|paragraph|list|table|other) / `semantics{scene,intent,entities,relations}`
/ `visual` (optional) / `uncertainty`; **no bbox, no per-line language, no
per-entity evidence, and lines/regions must not repeat full_text** — v2
dropped fabricated coordinates) → evidence file, ② a free-form **analysis**
answering your `prompt` verbatim (no JSON constraints; a Chinese default
question is used when no prompt is given). The analysis lands in the `analysis`
field of both the slim result and the evidence file (additive, backward
compatible — legacy v1 shapes with bbox/blocks still parse). If the analysis
request fails, the OCR evidence still succeeds (degraded, no `analysis`).

**Tool split (v4)**: `analyze_image` runs semantic structure
(`summary`/`layout.regions`/`semantics`/`uncertainty`, ModLens v2/v3 contract
minus ocr) **in parallel** with a free-form `analysis` of your prompt and
merges them into one evidence file. `ocr_image` is the pure-OCR service
(single request, `preview` + `full_text_chars` + `evidence_path`). Both tools
key evidence by the IMAGE sha256, so calling them in parallel for the same
image merges into ONE json file (`result` keeps `ocr` + `summary`/`layout`/
`semantics` + `analysis`); a write queue serializes concurrent merges. A
global API gate (max 5 concurrent vision requests) plus a one-shot retry
(3s backoff on 5xx/429/timeout) for analysis keep multi-image scenarios
stable.
if an endpoint (or a local proxy that forces streaming) still answers with
`text/event-stream`, the SSE payload is parsed transparently — full-completion
blocks and `delta` chunk streams are both supported, `[DONE]` is ignored.
Reasoning models whose `content` is `null` fall back to `reasoning_content`
only when it contains our JSON contract; otherwise a clear error is returned.

**Output length is not capped**: OpenAI-compatible providers (Zhipu, Gemini,
OpenAI, ...) receive no `max_tokens` field and use the model's own maximum
output — whatever the model returns is persisted to the evidence file and the
slim projection stays <32KB. Anthropic requires `max_tokens`, so it gets a
16384 fallback. Full-length model output can therefore never be cut off mid-JSON.

`ocr_image(image, format?)` → `{ image, format, evidence_path, preview, full_text_chars }`.

`compare_images(images[2-4], prompt?)` → per-image slim blocks with independent
`image` + `evidence_path`.

`find_image(filename?, sha256Prefix?, limit?)` → registry matches
(metadata only, never image bytes).

## Test

```bash
npm test    # 95 tests: config + direct (mock HTTP) + tools + real MCP protocol e2e
```

## Security

- No shell spawning; HTTP requests use argument-safe `fetch` with only the
  configured endpoint
- Errors are sanitized (keys / Bearer / Basic redacted) before reaching the
  model context; the config key is never echoed back
- Evidence files and the registry are written with 0600 / 0700 permissions
- The key lives either in `~/.reasonix/vision-config.json` (0600, not in
  version control) **or** in the Reasonix MCP panel env (`VISION_API_KEY`,
  persisted plaintext in the Reasonix config.toml — `/mcp` shows it redacted;
  keep that config out of shared screens and backups). Either way: rotate the
  key if it ever appears in logs or chats