Skip to main content
Glama

Shadow Vision

English | 简体中文

Give text-only LLMs a pair of eyes. Shadow Vision is an open-source MCP vision service that enables AI Agents to see, understand, and analyze real-world information through vision_ocr, vision_inspect, vision_annotate, vision_layout, vision_reconstruct, and vision_compare — without switching host models.

Why It's Different

  • MCP-Native: Compatible with Codex, Claude Desktop, Cursor, and other MCP clients

  • Pluggable Backends: Ollama, OpenAI-compatible, Anthropic, Gemini

  • Local-First: Keep images and inference entirely on your machine when using Ollama

  • Versatile Input: Supports local file paths, base64 image data, or remote HTTP(S) URLs

Related MCP server: OpenSight MCP

How It Works

The text model calls Shadow Vision tools via MCP. Shadow Vision forwards the image and prompt to the configured vision backend and returns the text result back to the model.

Quick Start

0. One-Click Run (No Clone Needed)

No need to clone the repository. Run directly:

uvx shadow-vision          # Python / uv users (recommended)
# Or for Node users
npx shadow-vision       # Requires uv installed locally

MCP configuration example:

[mcp_servers.vision]
command = "uvx"
args = ["shadow-vision"]
env = { VISION_BACKEND = "ollama", VISION_MODEL = "qwen3-vl:2b-instruct" }

npx shadow-vision is a thin wrapper that invokes uvx shadow-vision internally and requires uv installed on your machine. Both entry points behave identically.

1. Install from Source (Development / Self-Hosting)

Requires Python 3.11+ and uv:

git clone https://github.com/WardLu/shadow-vision.git
cd shadow-vision
uv sync

First install Ollama. If not running the Ollama desktop app, start the service manually:

ollama serve
ollama pull qwen3-vl:2b-instruct
ollama list

qwen3-vl:2b-instruct is the default vision model (non-thinking version, faster response). If you need stronger reasoning/thinking capability, switch to qwen3-vl:2b (thinking version); or change VISION_MODEL to any other vision model listed in ollama list.

3. Register as an MCP Service

Codex can run directly:

codex mcp add vision -- uv run shadow-vision

Or add to ~/.codex/config.toml:

[mcp_servers.vision]
type = "stdio"
command = "uv"
args = ["run", "shadow-vision"]
cwd = "/path/to/shadow-vision"
env = { VISION_BACKEND = "ollama", VISION_MODEL = "qwen3-vl:2b-instruct", OLLAMA_URL = "http://127.0.0.1:11434/api/chat" }

After restarting your MCP client, simply ask the model to "take a look at this image".

Switching Models and Backends

VISION_BACKEND determines how requests are routed, and VISION_MODEL specifies the vision model. Update the environment variables in your MCP configuration and restart your client.

Switch to a local model:

env = { VISION_BACKEND = "ollama", VISION_MODEL = "your-downloaded-vision-model", OLLAMA_URL = "http://127.0.0.1:11434/api/chat" }

Switch to an OpenAI-compatible service:

env = { VISION_BACKEND = "openai_compatible", VISION_MODEL = "provider-vision-model-name", OPENAI_API_BASE = "https://api.example.com/v1", OPENAI_API_KEY = "sk-...", OPENAI_MAX_TOKENS = "1024", OPENAI_MAX_TOKENS_FIELD = "max_tokens" }

OPENAI_* denotes the OpenAI Chat Completions compatible protocol, which also applies to LM Studio, vLLM, and any other service offering /v1/chat/completions.

Free vision example for Chinese platforms (Zhipu GLM-4V-Flash):

env = { VISION_BACKEND = "openai_compatible", VISION_MODEL = "glm-4v-flash", OPENAI_API_BASE = "https://open.bigmodel.cn/api/paas/v4", OPENAI_API_KEY = "your-zhipu-key" }

Other OpenAI-compatible providers only require modifying OPENAI_API_BASE and VISION_MODEL: SiliconFlow https://api.siliconflow.cn/v1, Alibaba Bailian https://dashscope.aliyuncs.com/compatible-mode/v1, StepFun https://api.stepfun.com/v1, Tencent Hunyuan https://api.hunyuan.cloud.tencent.com/v1, Moonshot https://api.moonshot.cn/v1, etc.

Privacy Notice: API backends (including third-party platforms) send image content as base64 to the respective provider's servers. For confidential or sensitive images, use the local ollama backend to prevent external data transfer.

Configuring Backends

General Variables

Variable

Default

Description

VISION_BACKEND

ollama

ollama / openai_compatible / anthropic / gemini

VISION_MODEL

qwen3-vl:2b-instruct

Vision model name

VISION_TIMEOUT

180

Read timeout (seconds), compatibility alias for VISION_READ_TIMEOUT

VISION_CONNECT_TIMEOUT

10

Connect timeout (seconds)

VISION_READ_TIMEOUT

180

Read timeout (seconds)

VISION_MAX_RETRIES

2

Retry attempts for transient / 5xx errors (total requests = 1 + this value)

VISION_RETRY_BASE_DELAY

1.0

Exponential backoff base delay in seconds

Advanced Configuration (Images & Security)

Variable

Default

Description

VISION_AUTO_COMPRESS

true

Whether to automatically compress large images

VISION_MAX_LONG_EDGE

1800

Compression threshold: long-edge pixels

VISION_MAX_PIXELS

3500000

Compression threshold: total pixels

VISION_COMPRESS_QUALITY

85

JPEG re-encoding quality

VISION_AUTO_TILE

true

Whether to automatically tile extra-long images

VISION_TILE_LONG_EDGE

3600

Tiling threshold: long-edge pixels

VISION_TILE_OVERLAP

100

Tiling overlap pixels

VISION_MAX_TILES

8

Maximum number of tiles per image

VISION_TASK_ROUTING

true

Whether to enable heuristic task routing for vision_inspect

VISION_ALLOW_REMOTE_URL

true

Whether to allow remote URL image inputs

VISION_MAX_REMOTE_SIZE

20971520

Maximum remote image size in bytes (20MB)

VISION_FETCH_TIMEOUT

30

Remote fetch timeout (seconds)

VISION_SSRF_ALLOW_PRIVATE

false

Whether to allow private / intranet addresses (strongly discouraged)

VISION_MAX_BATCH_IMAGES

5

Maximum number of images per vision_compare call

Ollama

Variable

Default

Description

OLLAMA_URL

http://127.0.0.1:11434/api/chat

Ollama chat endpoint

Run ollama pull <vision-model-name> before use to download the model.

OpenAI-compatible

Variable

Default

Description

OPENAI_API_BASE

http://127.0.0.1:11434/v1

Compatible service base URL

OPENAI_API_KEY

empty

Usually left blank for local services

OPENAI_MAX_TOKENS

unset

Optional max output tokens; omits token limit field when unset

OPENAI_MAX_TOKENS_FIELD

max_tokens

Optional: max_tokens or max_completion_tokens

Different providers support different token limit fields: use max_tokens for legacy fields, max_completion_tokens for newer APIs, or leave OPENAI_MAX_TOKENS unset if neither is accepted. Legacy environment variables VISION_API_BASE, VISION_API_KEY, VISION_MAX_TOKENS, and VISION_MAX_TOKENS_FIELD remain supported for backward compatibility.

Anthropic / Gemini

VISION_BACKEND=anthropic ANTHROPIC_API_KEY=sk-ant-... VISION_MODEL=your-claude-vision-model uv run shadow-vision
VISION_BACKEND=gemini GEMINI_API_KEY=AIza... VISION_MODEL=your-gemini-vision-model uv run shadow-vision

Anthropic also supports ANTHROPIC_BASE_URL, ANTHROPIC_VERSION, and ANTHROPIC_MAX_TOKENS; Gemini also supports GEMINI_BASE_URL and GEMINI_MAX_TOKENS.

Tools

vision_ocr

Extract text from screenshots, invoices/receipts, documents, or tables:

vision_ocr(image_path="/tmp/receipt.png")

vision_inspect

Describe an image, or answer questions about an image:

vision_inspect(image_path="/tmp/design.png", question="List any UI bugs you see.")

Both tools also support:

  • task: Optional task guidance (vision_ocr: general/error/table; vision_inspect: general/ui_structure/ui_bug/chart)

  • image_path: Server-readable local image path

  • image_base64 + mime_type: Base64-encoded image data

  • image_url: Remote HTTP(S) image URL (with automatic SSRF protection)

All image tools accept one of image_path / image_base64 / image_url as input, with precedence: image_base64 > image_path > image_url.

vision_annotate

Identify bounding boxes, arrows, underlines, highlights, strikethroughs, handwritten notes, and other user annotations. Outputs structured JSON with annotation → target relationships, types, bounding boxes (bbox), and confidence scores:

vision_annotate(image_path="/tmp/marked.png", focus="Explain changes in order of markup")

vision_layout

Analyze image and UI layout structure, outputting structured JSON with canvas, containers, element bounding boxes (bbox), typography styles, and element hierarchy:

vision_layout(image_path="/tmp/ui.png")

vision_reconstruct

Reconstruct screenshots into code (html / react / svg), generating markup along with model self-inspection. Optionally provide JSON from vision_layout as layout reference:

vision_reconstruct(image_path="/tmp/ui.png", target_format="html", reference_layout="<layout json>")

vision_compare

Analyze multiple related images in a single call (diff / compare / sequence), with optional per-image label for easy reference:

vision_compare(images=[{"image_path": "/tmp/a.png", "label": "before"}, {"image_path": "/tmp/b.png", "label": "after"}], task="diff")

Local Model Selection and Benchmarking

The Ollama model library lists download size, context window, and vision capabilities, but model package size does not represent minimum memory requirements. It is recommended to start with qwen3-vl:2b-instruct (non-thinking version, low latency); if OCR or complex chart comprehension is insufficient, benchmark against qwen3-vl:4b, qwen3-vl:8b, or document-OCR oriented minicpm-v4.5:q4_0.

Prepare 3–5 real-world images covering OCR, UI screenshots, charts, and hard edge cases, and compare models using identical prompts:

MODEL=qwen3-vl:2b-instruct
IMAGE=/absolute/path/to/test.png

time ollama run "$MODEL" "$IMAGE" "Transcribe all text from the image accurately, outputting only the text."
time ollama run "$MODEL" "$IMAGE" "Describe the image contents and list any areas of uncertainty."

Track OCR error count, accuracy of key objects and relationships, hallucinations, full round-trip latency, and processor status in ollama ps. Testing Ollama directly first before testing through vision_ocr / vision_inspect via MCP helps distinguish model capability limits from MCP configuration issues.

Recommended resources:

Supported Agents

All agents launch the same command: uv run shadow-vision.

Agent

Configuration File

Codex

~/.codex/config.toml

Claude Code

.mcp.json

Cursor

.cursor/mcp.json

VS Code Copilot

.vscode/mcp.json

Windsurf

.windsurf/mcp_config.json

Claude Desktop

claude_desktop_config.json

OpenCode

opencode.json

Development

uv sync
uv run python -c "import vision_mcp.server; print('ok')"
uv run pytest

Contact

If you are interested in B2B products, AI product development, supply chain digitization, or Shadow series products, feel free to get in touch:

  • X (Twitter): @Gollumgulu

  • WeChat Official Account: Ward 的 AI 产品实战

Open for 1-on-1 consulting and advisory: Product Diagnostics · AI Implementation · Workflows / Skills · Custom Solutions

License

MIT

Available Tools

6 tools
vision_annotateB

Identify user's visual annotations (circles, arrows, underlines, highlights, scribbles, handwritten text) and their targets, returning structured JSON.

ParametersJSON Schema
NameRequiredDescriptionDefault
focusNoUser note to help interpret annotation intent
modelNoVision model nameqwen3-vl:2b-instruct
image_urlNoRemote HTTP(S) image URL
mime_typeNoMIME type of image_base64 (e.g. image/png)
image_pathNoLocal path to the image file
image_base64NoBase64-encoded image data

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden and does not meet it. It discloses only that output is 'structured JSON'; it says nothing about whether the operation is read-only, which of the three image inputs (image_url / image_path / image_base64) is required or takes precedence, model/latency implications, or any failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler. The parenthetical enumeration is dense but every item earns its place by scoping what counts as an 'annotation'.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should say more about the 'structured JSON' it returns (shape/fields), and it never resolves the ambiguity of six optional params offering three alternate image inputs with nothing marked required. Adequate for a first-order call, but incomplete for reliable invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all six parameters including the model default and the MIME/Base64 pairing. The description adds nothing about parameters, so the baseline 3 applies — it neither compensates for nor undermines the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Identify') and resource ('user's visual annotations') and enumerates the annotation kinds (circles, arrows, underlines, highlights, scribbles, handwritten text) plus 'their targets'. This is clearly distinct from vision_ocr or vision_layout in practice, but no sibling is named, so differentiation is implicit rather than explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no stated preconditions, and no mention of when to prefer this over vision_ocr (which also handles handwritten text), vision_inspect, or vision_layout. Usage must be inferred entirely from the tool name and the enumeration of annotation types.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_compareC

Analyze multiple related images (diff, compare, sequence) in one call.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskNocompare
modelNoVision model nameqwen3-vl:2b-instruct
imagesNo
questionNoQuestion to ask about the imagesDescribe this image in detail.

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It says the analysis happens 'in one call', but discloses nothing about cost/latency of a multi-image model call, the 5-image cap, how labels/ordering affect results, or what the response contains. Significant gaps for a 4-param tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the core scope front-loaded and no filler. It is efficient, though arguably too terse for the omissions noted elsewhere.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema and no annotations, so the description would need to explain return shape, task semantics, and ordering. Instead it leaves the meaning of diff/compare/sequence, model selection, and result structure entirely unaddressed for a 4-param multi-image tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 50% schema description coverage, the description should compensate, but it only restates the task enum values already in the schema. It adds no meaning for 'model' (why/when to override the default), 'question', or the nested image object fields (label, image_path, image_base64, mime_type).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Analyze') and resource ('multiple related images') and enumerates the three task modes, which scopes it against the single-image siblings (vision_inspect, vision_ocr, vision_layout). It never names an alternative tool, but the multi-image framing is clear enough that an agent can distinguish it.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The parenthetical '(diff, compare, sequence)' implies the usage modes, but there is no explicit when-to-use versus vision_inspect or vision_annotate, and no guidance on how many images or what kind of relationship warrants this tool over single-image ones. Usage is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_inspectC

Describe or answer questions about an image. Pass image_path (local file) or image_base64 + mime_type.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskNoOptional inspect task hint
modelNoVision model nameqwen3-vl:2b-instruct
questionNoQuestion to ask about the imageDescribe this image in detail.
image_urlNoRemote HTTP(S) image URL
mime_typeNoMIME type of image_base64 (e.g. image/png)
image_pathNoLocal path to the image file
image_base64NoBase64-encoded image data

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden, and it delivers little beyond input mechanics. It never says what the tool returns (text answer), whether it is read-only, what happens on model failure, or why a default model is pre-selected — all relevant for a remote vision inference call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences, front-loaded with the capability before the input modes; nothing is padded. It loses a point only because the second sentence enumerates two of three input paths rather than being complete.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 7 parameters, no required fields, no annotations, and no output schema, the description is thin. It never explains the task enum (ui_structure, ui_bug, chart), the model override, or what happens if no image parameter is supplied.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds one genuinely useful semantic: that image_base64 must be paired with mime_type, and that image_path means a local file — but it omits image_url, a third input mode present in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb pair (describe / answer questions) and the resource (an image), so the core capability is unambiguous. It does not distinguish itself from siblings like vision_ocr, vision_annotate, or vision_compare, which an agent must differentiate between in this cluster.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to pick this tool over the five sibling vision_* tools, and no exclusions (e.g., 'use vision_ocr for text extraction'). The description only tells the agent how to supply an image, not when this tool is the right choice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_layoutC

Analyze an image's layout structure and return structured JSON (canvas, elements, relations).

ParametersJSON Schema
NameRequiredDescriptionDefault
taskNoLayout task hint
modelNoVision model nameqwen3-vl:2b-instruct
image_urlNoRemote HTTP(S) image URL
mime_typeNoMIME type of image_base64 (e.g. image/png)
image_pathNoLocal path to the image file
image_base64NoBase64-encoded image data

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the full behavioral burden. It says it analyzes and returns structured JSON, implying a read-only operation, but does not state whether it is read-only, whether it calls an external model, latency/cost implications, or any side effects. The output shape is lightly previewed, but key behavioral traits are missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that front-loads the action and result shape with no wasted words. It is appropriately sized for the tool's purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with no annotations and no output schema, the description is too thin. It partially covers the return structure but does not clarify how to supply the image (URL vs. path vs. base64+mime_type), whether any parameters are required, or what calling the tool implies operationally.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents each parameter (task, model, image_url, mime_type, image_path, image_base64). The description adds no additional meaning beyond the schema, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Analyze') and resource ('image's layout structure') and even previews the return structure (canvas, elements, relations), so an agent knows what the tool does. It does not explicitly differentiate from siblings like vision_ocr or vision_reconstruct, but the 'layout structure' focus is fairly distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus siblings such as vision_inspect, vision_ocr, or vision_reconstruct. It also omits prerequisites, when-not-to-use, or how to choose among the image input parameters.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_ocrC

Extract all text from an image. Pass image_path (local file) or image_base64 + mime_type.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskNoOptional OCR task hint
modelNoVision model nameqwen3-vl:2b-instruct
image_urlNoRemote HTTP(S) image URL
mime_typeNoMIME type of image_base64 (e.g. image/png)
image_pathNoLocal path to the image file
image_base64NoBase64-encoded image data

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full disclosure burden. It says text is extracted 'all' from the image, but omits error behavior, whether the optional task/model hints change output, and any auth or resource constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core action and followed by the input requirement. No filler, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 6-parameter tool with no output schema and no annotations, the description is too thin. It never mentions the image_url input path, the task enum, or the model override, leaving the agent to discover three of six parameters only from the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents every parameter and the baseline is 3. The description does add the useful implication that image_path and image_base64 are alternative input modes, but it omits the schema's image_url input entirely and says nothing about task or model.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Extract all text from an image', which unambiguously identifies OCR. It does not, however, differentiate itself from siblings like vision_inspect or vision_layout, so an agent gets no help distinguishing this tool's niche from the other vision tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains how to supply the image (image_path or image_base64 + mime_type), but that is parameter syntax, not usage guidance. There is no statement of when to choose OCR over vision_inspect or vision_layout, and no exclusions or prerequisites are given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_reconstructC

Reconstruct a screenshot into code (html/react/svg) with a model self-check.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNoVision model nameqwen3-vl:2b-instruct
renderNoAttempt closed-loop rendering (default from VISION_RECONSTRUCT_RENDER)
image_urlNoRemote HTTP(S) image URL
mime_typeNoMIME type of image_base64 (e.g. image/png)
image_pathNoLocal path to the image file
iterationsNoMax refinement iterations (default from config)
image_base64NoBase64-encoded image data
target_formatNohtml
reference_layoutNoOptional JSON output from vision_layout

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the model self-check loop, but says nothing about cost/latency of vision model inference, whether files are written to disk (render implies a side effect), or what happens when iterations run out. For an unannotated tool with a rendering step, this is thin.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence with no filler; the purpose and output formats come first. It is efficient, though arguably terse for a nine-parameter tool, so it is not a model of expressive economy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Nine parameters, zero required, no output schema, and no annotations — the definition should at least clarify that exactly one of image_url/image_path/image_base64 must be supplied, and what a successful result looks like. Defaults for render/iterations are deferred to config variables without explanation. The single sentence is not enough to call this correctly without opening the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 89%, so the schema already documents model, render, iterations, mime_type, image sources, and reference_layout. The description adds no parameter-level meaning beyond the format list that is already an enum in the schema. Baseline 3 applies when the schema does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a concrete verb+resource+output: 'Reconstruct a screenshot into code (html/react/svg) with a model self-check.' The output formats distinguish it from siblings like vision_ocr and vision_layout, which produce text or layout instead of code. It stops short of naming an alternative explicitly, so it is clear but not fully differentiated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no statement of when to reach for this tool versus vision_layout, vision_ocr, or vision_compare, which is the most likely confusion given the sibling set. The only hints are 'screenshot into code' and the optional reference_layout parameter, leaving the routing decision to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv0.1.1
    • First observedvision_annotate
    • First observedvision_compare
    • First observedvision_inspect
    • First observedvision_layout
    • First observedvision_ocr
    • First observedvision_reconstruct

TDQS

B3.4/5.0

Scored across 6 tools

Disambiguation4/5

Each tool targets a distinct output modality: description, annotations, OCR text, layout JSON, code reconstruction, and multi-image comparison. Some boundary blur exists between vision_inspect (general description, which may incidentally read text) and vision_ocr, and between vision_layout and vision_reconstruct, but the descriptions clarify the intended scope of each.

Naming Consistency5/5

All six tools follow a strict vision_<verb> snake_case pattern (inspect, annotate, ocr, layout, reconstruct, compare). The convention is predictable and readable throughout.

Tool Count5/5

Six tools is well-scoped for a vision-analysis server, with each tool earning its place by covering a distinct capability. No redundant or filler tools are present.

Completeness4/5

The surface covers the major analysis workflows: description, OCR, annotation detection, layout parsing, code reconstruction, and multi-image comparison. Minor gaps like detecting/returning bounding boxes or image generation/editing exist, but they are outside the apparent analysis-focused purpose.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    B
    maintenance
    Local vision-capable MCP server that lets AI agents describe screenshots, UI, charts, and photos via vision and OCR tools, with support for multiple providers and automatic fallback.
    5
    MIT
  • A
    license
    A
    quality
    B
    maintenance
    Provides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.
    3
    6 npm
    MIT