manga-vision-mcp

English | 日本語

A local MCP server (stdio) backed by any OpenAI-compatible vision VLM. Primary use case: checking whether generated images conform to the composition / scenery / expression specs derived from the scenario (the generation spec — the promptEn/sceneEn of the name (storyboard) — is the contract). Besides conformance checks, a mode switch selects best-frame candidate review (frame), reference checking (refcheck), or rubric scoring (rubric).

It has three VLM slots (primary / nsfw / fallback), each taking its own BASE_URL / MODEL / API_KEY — beyond NanoGPT, any OpenAI-compatible provider ((baseUrl)/chat/completions + Bearer auth) works: OpenRouter, OpenAI, Google, local vLLM, etc. With nothing configured, it defaults to NanoGPT z-ai/glm-5.3-flash-uncensored as before.

Tools

The difference from URL-only remote MCPs (4_5v_mcp etc.) is that local files can be passed directly. Images are embedded as base64 data URLs (avoiding remote-fetch failures on the API side).

VLM slots and routing (primary / nsfw / fallback)

Set three variables per slot in .env (or real env vars / MCP config env):

• Backward compatible: the legacy NANOGPT_API_KEY / NANOGPT_BASE_URL / NANOGPT_MODEL still work as substitutes when PRIMARY_* is unset (PRIMARY_* wins)

• The tool argument vlm ('primary' / 'nsfw' / 'fallback', default primary) explicitly selects a slot (explicit fallback selection is handy for connectivity checks etc.)

• When automatic fallback kicks in, the response carries [fallback: <model>] and the log records fallbackFrom (failed slot + error)

• Errors before the API stage (e.g. image load failure) do not trigger fallback (API-stage failures only)

Setup

cd manga-vision-mcp
npm install
cp .env.example .env   # put your primary-slot API key etc. in .env (git-ignored)
npm run smoke          # sanity check (verifies up to tools/list even without a key)

Register with ZCode via mcp.servers in .zcode/config.json (git-ignored, so paths and settings are local-only). A session restart is required after registration. Tool names: mcp__manga-vision-mcp__analyze_image / mcp__manga-vision-mcp__compare_images.

Resolution constraints (token & latency savings)

Image tokens scale with pixel count (measured ~1,300 tokens/MP, no provider-side downsampling), and latency tracks image tokens. By default images are downscaled to a long edge of 1024px before sending (lanczos3, withoutEnlargement. JPEGs are recompressed at q92; everything else stays PNG — so recompression artifacts never create phantom defects).

• VISION_MAX_LONG_EDGE in .env (default 1024, 0 = unlimited)

• Override per call with the max_long_edge parameter (0 = unlimited). Raise it only when inspecting hand defects, stray text, or linework/tone detail (finger counts, small text, and screentone crushing are invisible at low resolution)

Measured (832×1280 source): full size 1401 tokens / 7.4s → long edge 1024: 909 / 6.6s → 768: 525 / 4.0s. Responses include a [resized: 832x1280→665x1024] record.

Inspection modes (mode)

The default is conformance (the primary use case). Every mode always prepends the same baseline (only facts verifiable from the image · explicitly state what cannot be determined · cite evidence with in-image positions) — to curb review variance.

• conformance (default, both tools): conformance check against the generation spec — 〇△✕ + evidence on the three axes (composition / scenery / expression), plus a list of 【Missing】 (the spec requires it but the image lacks it) and 【Invented】 (drawn even though the spec doesn't ask for it)

• frame (analyze_image): best-frame candidate review — three layers of acting (mid-action · gaze target · direction of force) / known failure modes (hallucinated text, speech bubbles, panel borders · broken fingers · distorted faces · merged people) / composition (camera height and angle)

• refcheck (compare_images): reference check — Image A as ground truth, per-part 〇△✕ (hair / eyes / tops / bottoms / accessories) · background layout · spatial relations between people (adjacent seating · facing each other · distance · partitions) · list of "inventions" absent from A

• rubric (both): rubric scoring — 5-point scale, pass line 4, conservative scoring, one improvement lever per axis below 4. Scoring axes are supplied via the prompt

• free: no scaffold (raw prompt passthrough)

What to put in the prompt (calling convention for conformance checks)

The conformance standard is "the instructions actually passed to the generation engine" — paraphrasing shifts the inspection baseline:

• Quote the generation spec verbatim: quote promptEn (danbooru-style) or sceneEn (H3-style) as-is. Also append auto-concatenated tags such as camera/angle and character-count tags

• Intent supplement: subject / direction (the hook→response design — what the panel was trying to depict)

• Dialogue: the panel's dialogue and its type (needed to check mouth openness in speaking panels, and speech-bubble vs. expression mismatch)

• What the image is: one word — frame candidate (H3-extracted, unfinished) / finalized image (post-i2i) / character sheet

• Scoring axes: the axes to score in rubric mode (e.g. intent conveyed / character consistency / cohesion / technique)

Example: "Frame candidate (H3-extracted, unfinished). Generation spec (promptEn, verbatim): "closeup of a small hand pressing the enter key on a black keyboard, monitor glow lighting the fingertips, dark room". direction: the hook is the keystroke itself; the result is not shown yet. Dialogue: narration ナレ『わたしの夢は、漫画を描くこと。』 ("My dream is to draw manga." — hand close-up, no speech bubble)"

Constraints / specs

• Endpoint: <BASE_URL>/chat/completions per slot (OpenAI-compatible, Bearer auth). Primary default: https://nano-gpt.com/api/v1/chat/completions

• Supported formats: PNG / JPEG / WebP only. 16MB per image, ~24MB total per request (safety margin against the 32MB API limit)

• Responses end with [tokens: prompt=X completion=Y / Zms] (for measuring cost and latency)

• Model and BASE_URL are per-slot .env settings (*_VLM_MODEL / *_VLM_BASE_URL)

• The API key lives in manga-vision-mcp/.env (default) or real env vars (which take priority)

Logging (accuracy & error analysis)

Every tool call is appended to a daily JSONL log (logs/YYYY-MM-DD.jsonl, one line per call). Enabled by default. Prompts, system scaffolds, and full response texts are recorded; images are metadata-only (path, MIME, bytes, pre/post-resize dimensions) — base64 payloads and API keys are never written. Both successes and failures (image load failure / HTTP error / JSON parse failure) are logged.

• Disable: VISION_LOG=0 in .env

• Change destination: VISION_LOG_DIR=/path/to/logs (default: logs/ at the repo root, git-ignored)

• Note: schema violations in tool input (zod validation) are rejected by the SDK before the handler runs, so they never reach the log

Key entry fields: ts / seq / tool / mode / vlm (slot used) / model (that slot's model) / fallbackFrom (only on automatic fallback: failed slot, model, error) / params (maxLongEdge, maxTokens) / prompt (full text) / system (full scaffold) / images[] (tag=A,B,… · label · mime · bytes · origDims · sentDims) / response (full text · promptTokens · completionTokens · finishReason · latencyMs) / error (stage=load|api · message · httpStatus · body). Exactly one of response / error is present.

Analysis examples:

cd logs
jq -c 'select(.error != null) | {ts, tool, err: .error.message}' *.jsonl      # list errors
jq -r '.response.latencyMs' *.jsonl | sort -n | awk '{a[NR]=$1} END {print "p50", a[int(NR/2)], "max", a[NR]}'   # latency distribution
jq -r '[.ts, .mode, (.response.completionTokens // 0), .response.text] | @tsv' *.jsonl   # accuracy audit (response texts side by side)
jq -r 'select(.mode=="conformance") | .response.text' *.jsonl | grep -c '【判定】準拠$'  # conformance-rate tally

Re-inspections of the same panel (comparing prompts or max_long_edge) can be correlated via images[].label.

Latency and timeouts

Vision responses generate at ~20–35 tokens/s, so detailed inspections take 30–60 seconds (measured: 29–35s for a single detailed inspection, 35s for a detailed 3-image refcheck). Harness-side MCP timeout defaults are around 30 seconds, so raise them at registration:

• ZCode: add "timeoutMs": 130000 to the server definition (mcp.servers in .zcode/config.json; restart the session to apply)

• MCP SDK direct clients: callTool defaults to a 60s timeout — raise it via options.timeout

• Claude Code: extend with the MCP_TOOL_TIMEOUT environment variable

Guidance for the agent side: a timeout failure is "insufficient configuration", not "a wrong call". Multi-image and detailed rubric runs normally take tens of seconds.

Using with other coding harnesses

The server is a stdio MCP, harness-agnostic. Any client can spawn node index.mjs (from the repo root; node>=20, reusing the npm installed node_modules). The server reads .env itself, so no secrets need to go into each harness's config. Each connection gets its own process, so multiple harnesses and sessions can use it concurrently without conflicts. It works from any other repo via absolute paths (image paths are passed per call, so it isn't tied to a repo).

When the same name is registered in multiple scopes, precedence follows each harness's rules (e.g. ZCode: CLI > env vars > user scope > workspace scope). With the same name, only the top scope connects and lower definitions are ignored (no double connection).

ZCode

• Workspace: .zcode/config.json → mcp.servers

• All workspaces: ~/.zcode/cli/config.json → mcp.servers

Claude Code

claude mcp add manga-vision-mcp --scope user -- node /path/to/manga-vision-mcp/index.mjs

For project scope, use .mcp.json at the repo root (${CLAUDE_PROJECT_DIR} makes the path portable):

{ "mcpServers": { "manga-vision-mcp": { "command": "node", "args": ["${CLAUDE_PROJECT_DIR}/index.mjs"] } } }

Codex (~/.codex/config.toml)

[mcp_servers.manga-vision-mcp]
command = "node"
args = ["/path/to/manga-vision-mcp/index.mjs"]

Cursor (project: .cursor/mcp.json, global: ~/.cursor/mcp.json)

{ "mcpServers": { "manga-vision-mcp": { "command": "node", "args": ["/path/to/manga-vision-mcp/index.mjs"] } } }

License

MIT — see LICENSE.