shadow-vision
Allows local-first vision processing through a local Ollama server, using downloaded vision models for OCR, inspection, annotation, layout analysis, reconstruction, and comparison.
Allows routing vision requests to OpenAI-compatible Chat Completions services, including OpenAI and compatible providers, for OCR, inspection, annotation, layout analysis, reconstruction, and comparison.
Shadow Vision
English | 简体中文
Give text-only LLMs a pair of eyes. Shadow Vision is an open-source MCP vision service that enables AI Agents to see, understand, and analyze real-world information through vision_ocr, vision_inspect, vision_annotate, vision_layout, vision_reconstruct, and vision_compare — without switching host models.
Why It's Different
MCP-Native: Compatible with Codex, Claude Desktop, Cursor, and other MCP clients
Pluggable Backends: Ollama, OpenAI-compatible, Anthropic, Gemini
Local-First: Keep images and inference entirely on your machine when using Ollama
Versatile Input: Supports local file paths, base64 image data, or remote HTTP(S) URLs
Related MCP server: OpenSight MCP
How It Works
The text model calls Shadow Vision tools via MCP. Shadow Vision forwards the image and prompt to the configured vision backend and returns the text result back to the model.
Quick Start
0. One-Click Run (No Clone Needed)
No need to clone the repository. Run directly:
uvx shadow-vision # Python / uv users (recommended)
# Or for Node users
npx shadow-vision # Requires uv installed locallyMCP configuration example:
[mcp_servers.vision]
command = "uvx"
args = ["shadow-vision"]
env = { VISION_BACKEND = "ollama", VISION_MODEL = "qwen3-vl:2b-instruct" }
npx shadow-visionis a thin wrapper that invokesuvx shadow-visioninternally and requires uv installed on your machine. Both entry points behave identically.
1. Install from Source (Development / Self-Hosting)
Requires Python 3.11+ and uv:
git clone https://github.com/WardLu/shadow-vision.git
cd shadow-vision
uv sync2. Use Local Ollama (Recommended for Beginners)
First install Ollama. If not running the Ollama desktop app, start the service manually:
ollama serve
ollama pull qwen3-vl:2b-instruct
ollama listqwen3-vl:2b-instruct is the default vision model (non-thinking version, faster response). If you need stronger reasoning/thinking capability, switch to qwen3-vl:2b (thinking version); or change VISION_MODEL to any other vision model listed in ollama list.
3. Register as an MCP Service
Codex can run directly:
codex mcp add vision -- uv run shadow-visionOr add to ~/.codex/config.toml:
[mcp_servers.vision]
type = "stdio"
command = "uv"
args = ["run", "shadow-vision"]
cwd = "/path/to/shadow-vision"
env = { VISION_BACKEND = "ollama", VISION_MODEL = "qwen3-vl:2b-instruct", OLLAMA_URL = "http://127.0.0.1:11434/api/chat" }After restarting your MCP client, simply ask the model to "take a look at this image".
Switching Models and Backends
VISION_BACKEND determines how requests are routed, and VISION_MODEL specifies the vision model. Update the environment variables in your MCP configuration and restart your client.
Switch to a local model:
env = { VISION_BACKEND = "ollama", VISION_MODEL = "your-downloaded-vision-model", OLLAMA_URL = "http://127.0.0.1:11434/api/chat" }Switch to an OpenAI-compatible service:
env = { VISION_BACKEND = "openai_compatible", VISION_MODEL = "provider-vision-model-name", OPENAI_API_BASE = "https://api.example.com/v1", OPENAI_API_KEY = "sk-...", OPENAI_MAX_TOKENS = "1024", OPENAI_MAX_TOKENS_FIELD = "max_tokens" }OPENAI_* denotes the OpenAI Chat Completions compatible protocol, which also applies to LM Studio, vLLM, and any other service offering /v1/chat/completions.
Free vision example for Chinese platforms (Zhipu GLM-4V-Flash):
env = { VISION_BACKEND = "openai_compatible", VISION_MODEL = "glm-4v-flash", OPENAI_API_BASE = "https://open.bigmodel.cn/api/paas/v4", OPENAI_API_KEY = "your-zhipu-key" }Other OpenAI-compatible providers only require modifying OPENAI_API_BASE and VISION_MODEL: SiliconFlow https://api.siliconflow.cn/v1, Alibaba Bailian https://dashscope.aliyuncs.com/compatible-mode/v1, StepFun https://api.stepfun.com/v1, Tencent Hunyuan https://api.hunyuan.cloud.tencent.com/v1, Moonshot https://api.moonshot.cn/v1, etc.
Privacy Notice: API backends (including third-party platforms) send image content as base64 to the respective provider's servers. For confidential or sensitive images, use the local
ollamabackend to prevent external data transfer.
Configuring Backends
General Variables
Variable | Default | Description |
|
|
|
|
| Vision model name |
|
| Read timeout (seconds), compatibility alias for |
|
| Connect timeout (seconds) |
|
| Read timeout (seconds) |
|
| Retry attempts for transient / 5xx errors (total requests = 1 + this value) |
|
| Exponential backoff base delay in seconds |
Advanced Configuration (Images & Security)
Variable | Default | Description |
|
| Whether to automatically compress large images |
|
| Compression threshold: long-edge pixels |
|
| Compression threshold: total pixels |
|
| JPEG re-encoding quality |
|
| Whether to automatically tile extra-long images |
|
| Tiling threshold: long-edge pixels |
|
| Tiling overlap pixels |
|
| Maximum number of tiles per image |
|
| Whether to enable heuristic task routing for |
|
| Whether to allow remote URL image inputs |
|
| Maximum remote image size in bytes (20MB) |
|
| Remote fetch timeout (seconds) |
|
| Whether to allow private / intranet addresses (strongly discouraged) |
|
| Maximum number of images per |
Ollama
Variable | Default | Description |
|
| Ollama chat endpoint |
Run ollama pull <vision-model-name> before use to download the model.
OpenAI-compatible
Variable | Default | Description |
|
| Compatible service base URL |
| empty | Usually left blank for local services |
| unset | Optional max output tokens; omits token limit field when unset |
|
| Optional: |
Different providers support different token limit fields: use max_tokens for legacy fields, max_completion_tokens for newer APIs, or leave OPENAI_MAX_TOKENS unset if neither is accepted. Legacy environment variables VISION_API_BASE, VISION_API_KEY, VISION_MAX_TOKENS, and VISION_MAX_TOKENS_FIELD remain supported for backward compatibility.
Anthropic / Gemini
VISION_BACKEND=anthropic ANTHROPIC_API_KEY=sk-ant-... VISION_MODEL=your-claude-vision-model uv run shadow-vision
VISION_BACKEND=gemini GEMINI_API_KEY=AIza... VISION_MODEL=your-gemini-vision-model uv run shadow-visionAnthropic also supports ANTHROPIC_BASE_URL, ANTHROPIC_VERSION, and ANTHROPIC_MAX_TOKENS; Gemini also supports GEMINI_BASE_URL and GEMINI_MAX_TOKENS.
Tools
vision_ocr
Extract text from screenshots, invoices/receipts, documents, or tables:
vision_ocr(image_path="/tmp/receipt.png")vision_inspect
Describe an image, or answer questions about an image:
vision_inspect(image_path="/tmp/design.png", question="List any UI bugs you see.")Both tools also support:
task: Optional task guidance (vision_ocr:general/error/table;vision_inspect:general/ui_structure/ui_bug/chart)image_path: Server-readable local image pathimage_base64+mime_type: Base64-encoded image dataimage_url: Remote HTTP(S) image URL (with automatic SSRF protection)
All image tools accept one of image_path / image_base64 / image_url as input, with precedence: image_base64 > image_path > image_url.
vision_annotate
Identify bounding boxes, arrows, underlines, highlights, strikethroughs, handwritten notes, and other user annotations. Outputs structured JSON with annotation → target relationships, types, bounding boxes (bbox), and confidence scores:
vision_annotate(image_path="/tmp/marked.png", focus="Explain changes in order of markup")vision_layout
Analyze image and UI layout structure, outputting structured JSON with canvas, containers, element bounding boxes (bbox), typography styles, and element hierarchy:
vision_layout(image_path="/tmp/ui.png")vision_reconstruct
Reconstruct screenshots into code (html / react / svg), generating markup along with model self-inspection. Optionally provide JSON from vision_layout as layout reference:
vision_reconstruct(image_path="/tmp/ui.png", target_format="html", reference_layout="<layout json>")vision_compare
Analyze multiple related images in a single call (diff / compare / sequence), with optional per-image label for easy reference:
vision_compare(images=[{"image_path": "/tmp/a.png", "label": "before"}, {"image_path": "/tmp/b.png", "label": "after"}], task="diff")Local Model Selection and Benchmarking
The Ollama model library lists download size, context window, and vision capabilities, but model package size does not represent minimum memory requirements. It is recommended to start with qwen3-vl:2b-instruct (non-thinking version, low latency); if OCR or complex chart comprehension is insufficient, benchmark against qwen3-vl:4b, qwen3-vl:8b, or document-OCR oriented minicpm-v4.5:q4_0.
Prepare 3–5 real-world images covering OCR, UI screenshots, charts, and hard edge cases, and compare models using identical prompts:
MODEL=qwen3-vl:2b-instruct
IMAGE=/absolute/path/to/test.png
time ollama run "$MODEL" "$IMAGE" "Transcribe all text from the image accurately, outputting only the text."
time ollama run "$MODEL" "$IMAGE" "Describe the image contents and list any areas of uncertainty."Track OCR error count, accuracy of key objects and relationships, hallucinations, full round-trip latency, and processor status in ollama ps. Testing Ollama directly first before testing through vision_ocr / vision_inspect via MCP helps distinguish model capability limits from MCP configuration issues.
Recommended resources:
Supported Agents
All agents launch the same command: uv run shadow-vision.
Agent | Configuration File |
Codex |
|
Claude Code |
|
Cursor |
|
VS Code Copilot |
|
Windsurf |
|
Claude Desktop |
|
OpenCode |
|
Development
uv sync
uv run python -c "import vision_mcp.server; print('ok')"
uv run pytestContact
If you are interested in B2B products, AI product development, supply chain digitization, or Shadow series products, feel free to get in touch:
X (Twitter): @Gollumgulu
WeChat Official Account: Ward 的 AI 产品实战
Xiaohongshu / Weibo / Douyin: "Ward 的 AI 产品实战" across platforms — Xiaohongshu · Weibo · Douyin
Product Homepage: Shadow Nexus
Email: wardlu@126.com
Open for 1-on-1 consulting and advisory: Product Diagnostics · AI Implementation · Workflows / Skills · Custom Solutions
License
MIT
Available Tools
6 toolsvision_annotateB
Identify user's visual annotations (circles, arrows, underlines, highlights, scribbles, handwritten text) and their targets, returning structured JSON.
| Name | Required | Description | Default |
|---|---|---|---|
| focus | No | User note to help interpret annotation intent | |
| model | No | Vision model name | qwen3-vl:2b-instruct |
| image_url | No | Remote HTTP(S) image URL | |
| mime_type | No | MIME type of image_base64 (e.g. image/png) | |
| image_path | No | Local path to the image file | |
| image_base64 | No | Base64-encoded image data |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden and does not meet it. It discloses only that output is 'structured JSON'; it says nothing about whether the operation is read-only, which of the three image inputs (image_url / image_path / image_base64) is required or takes precedence, model/latency implications, or any failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler. The parenthetical enumeration is dense but every item earns its place by scoping what counts as an 'annotation'.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should say more about the 'structured JSON' it returns (shape/fields), and it never resolves the ambiguity of six optional params offering three alternate image inputs with nothing marked required. Adequate for a first-order call, but incomplete for reliable invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all six parameters including the model default and the MIME/Base64 pairing. The description adds nothing about parameters, so the baseline 3 applies — it neither compensates for nor undermines the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Identify') and resource ('user's visual annotations') and enumerates the annotation kinds (circles, arrows, underlines, highlights, scribbles, handwritten text) plus 'their targets'. This is clearly distinct from vision_ocr or vision_layout in practice, but no sibling is named, so differentiation is implicit rather than explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no stated preconditions, and no mention of when to prefer this over vision_ocr (which also handles handwritten text), vision_inspect, or vision_layout. Usage must be inferred entirely from the tool name and the enumeration of annotation types.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
vision_compareC
Analyze multiple related images (diff, compare, sequence) in one call.
| Name | Required | Description | Default |
|---|---|---|---|
| task | No | compare | |
| model | No | Vision model name | qwen3-vl:2b-instruct |
| images | No | ||
| question | No | Question to ask about the images | Describe this image in detail. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It says the analysis happens 'in one call', but discloses nothing about cost/latency of a multi-image model call, the 5-image cap, how labels/ordering affect results, or what the response contains. Significant gaps for a 4-param tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with the core scope front-loaded and no filler. It is efficient, though arguably too terse for the omissions noted elsewhere.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema and no annotations, so the description would need to explain return shape, task semantics, and ordering. Instead it leaves the meaning of diff/compare/sequence, model selection, and result structure entirely unaddressed for a 4-param multi-image tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 50% schema description coverage, the description should compensate, but it only restates the task enum values already in the schema. It adds no meaning for 'model' (why/when to override the default), 'question', or the nested image object fields (label, image_path, image_base64, mime_type).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Analyze') and resource ('multiple related images') and enumerates the three task modes, which scopes it against the single-image siblings (vision_inspect, vision_ocr, vision_layout). It never names an alternative tool, but the multi-image framing is clear enough that an agent can distinguish it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The parenthetical '(diff, compare, sequence)' implies the usage modes, but there is no explicit when-to-use versus vision_inspect or vision_annotate, and no guidance on how many images or what kind of relationship warrants this tool over single-image ones. Usage is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
vision_inspectC
Describe or answer questions about an image. Pass image_path (local file) or image_base64 + mime_type.
| Name | Required | Description | Default |
|---|---|---|---|
| task | No | Optional inspect task hint | |
| model | No | Vision model name | qwen3-vl:2b-instruct |
| question | No | Question to ask about the image | Describe this image in detail. |
| image_url | No | Remote HTTP(S) image URL | |
| mime_type | No | MIME type of image_base64 (e.g. image/png) | |
| image_path | No | Local path to the image file | |
| image_base64 | No | Base64-encoded image data |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden, and it delivers little beyond input mechanics. It never says what the tool returns (text answer), whether it is read-only, what happens on model failure, or why a default model is pre-selected — all relevant for a remote vision inference call.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences, front-loaded with the capability before the input modes; nothing is padded. It loses a point only because the second sentence enumerates two of three input paths rather than being complete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 7 parameters, no required fields, no annotations, and no output schema, the description is thin. It never explains the task enum (ui_structure, ui_bug, chart), the model override, or what happens if no image parameter is supplied.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds one genuinely useful semantic: that image_base64 must be paired with mime_type, and that image_path means a local file — but it omits image_url, a third input mode present in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb pair (describe / answer questions) and the resource (an image), so the core capability is unambiguous. It does not distinguish itself from siblings like vision_ocr, vision_annotate, or vision_compare, which an agent must differentiate between in this cluster.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to pick this tool over the five sibling vision_* tools, and no exclusions (e.g., 'use vision_ocr for text extraction'). The description only tells the agent how to supply an image, not when this tool is the right choice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
vision_layoutC
Analyze an image's layout structure and return structured JSON (canvas, elements, relations).
| Name | Required | Description | Default |
|---|---|---|---|
| task | No | Layout task hint | |
| model | No | Vision model name | qwen3-vl:2b-instruct |
| image_url | No | Remote HTTP(S) image URL | |
| mime_type | No | MIME type of image_base64 (e.g. image/png) | |
| image_path | No | Local path to the image file | |
| image_base64 | No | Base64-encoded image data |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the full behavioral burden. It says it analyzes and returns structured JSON, implying a read-only operation, but does not state whether it is read-only, whether it calls an external model, latency/cost implications, or any side effects. The output shape is lightly previewed, but key behavioral traits are missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that front-loads the action and result shape with no wasted words. It is appropriately sized for the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter tool with no annotations and no output schema, the description is too thin. It partially covers the return structure but does not clarify how to supply the image (URL vs. path vs. base64+mime_type), whether any parameters are required, or what calling the tool implies operationally.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents each parameter (task, model, image_url, mime_type, image_path, image_base64). The description adds no additional meaning beyond the schema, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Analyze') and resource ('image's layout structure') and even previews the return structure (canvas, elements, relations), so an agent knows what the tool does. It does not explicitly differentiate from siblings like vision_ocr or vision_reconstruct, but the 'layout structure' focus is fairly distinct.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus siblings such as vision_inspect, vision_ocr, or vision_reconstruct. It also omits prerequisites, when-not-to-use, or how to choose among the image input parameters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
vision_ocrC
Extract all text from an image. Pass image_path (local file) or image_base64 + mime_type.
| Name | Required | Description | Default |
|---|---|---|---|
| task | No | Optional OCR task hint | |
| model | No | Vision model name | qwen3-vl:2b-instruct |
| image_url | No | Remote HTTP(S) image URL | |
| mime_type | No | MIME type of image_base64 (e.g. image/png) | |
| image_path | No | Local path to the image file | |
| image_base64 | No | Base64-encoded image data |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full disclosure burden. It says text is extracted 'all' from the image, but omits error behavior, whether the optional task/model hints change output, and any auth or resource constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core action and followed by the input requirement. No filler, no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 6-parameter tool with no output schema and no annotations, the description is too thin. It never mentions the image_url input path, the task enum, or the model override, leaving the agent to discover three of six parameters only from the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents every parameter and the baseline is 3. The description does add the useful implication that image_path and image_base64 are alternative input modes, but it omits the schema's image_url input entirely and says nothing about task or model.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Extract all text from an image', which unambiguously identifies OCR. It does not, however, differentiate itself from siblings like vision_inspect or vision_layout, so an agent gets no help distinguishing this tool's niche from the other vision tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to supply the image (image_path or image_base64 + mime_type), but that is parameter syntax, not usage guidance. There is no statement of when to choose OCR over vision_inspect or vision_layout, and no exclusions or prerequisites are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
vision_reconstructC
Reconstruct a screenshot into code (html/react/svg) with a model self-check.
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Vision model name | qwen3-vl:2b-instruct |
| render | No | Attempt closed-loop rendering (default from VISION_RECONSTRUCT_RENDER) | |
| image_url | No | Remote HTTP(S) image URL | |
| mime_type | No | MIME type of image_base64 (e.g. image/png) | |
| image_path | No | Local path to the image file | |
| iterations | No | Max refinement iterations (default from config) | |
| image_base64 | No | Base64-encoded image data | |
| target_format | No | html | |
| reference_layout | No | Optional JSON output from vision_layout |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the model self-check loop, but says nothing about cost/latency of vision model inference, whether files are written to disk (render implies a side effect), or what happens when iterations run out. For an unannotated tool with a rendering step, this is thin.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence with no filler; the purpose and output formats come first. It is efficient, though arguably terse for a nine-parameter tool, so it is not a model of expressive economy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Nine parameters, zero required, no output schema, and no annotations — the definition should at least clarify that exactly one of image_url/image_path/image_base64 must be supplied, and what a successful result looks like. Defaults for render/iterations are deferred to config variables without explanation. The single sentence is not enough to call this correctly without opening the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 89%, so the schema already documents model, render, iterations, mime_type, image sources, and reference_layout. The description adds no parameter-level meaning beyond the format list that is already an enum in the schema. Baseline 3 applies when the schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a concrete verb+resource+output: 'Reconstruct a screenshot into code (html/react/svg) with a model self-check.' The output formats distinguish it from siblings like vision_ocr and vision_layout, which produce text or layout instead of code. It stops short of naming an alternative explicitly, so it is clear but not fully differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no statement of when to reach for this tool versus vision_layout, vision_ocr, or vision_compare, which is the most likely confusion given the sibling set. The only hints are 'screenshot into code' and the optional reference_layout parameter, leaving the routing decision to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
6 tool updates
v0.1.1- First observed
vision_annotate - First observed
vision_compare - First observed
vision_inspect - First observed
vision_layout - First observed
vision_ocr - First observed
vision_reconstruct
TDQS
Scored across 6 tools
Each tool targets a distinct output modality: description, annotations, OCR text, layout JSON, code reconstruction, and multi-image comparison. Some boundary blur exists between vision_inspect (general description, which may incidentally read text) and vision_ocr, and between vision_layout and vision_reconstruct, but the descriptions clarify the intended scope of each.
All six tools follow a strict vision_<verb> snake_case pattern (inspect, annotate, ocr, layout, reconstruct, compare). The convention is predictable and readable throughout.
Six tools is well-scoped for a vision-analysis server, with each tool earning its place by covering a distinct capability. No redundant or filler tools are present.
The surface covers the major analysis workflows: description, OCR, annotation detection, layout parsing, code reconstruction, and multi-image comparison. Minor gaps like detecting/returning bounding boxes or image generation/editing exist, but they are outside the apparent analysis-focused purpose.
Maintenance
Related MCP Connectors
MCP server for visual regression testing: triage a PR's UI diffs from your coding agent.
OCR, transcription, file extraction, and image generation for AI agents via MCP.
MCP server for Qwen Image 3 AI image generation
MCP server for building and testing AI agents with multi-model experimentation and insights.
Related MCP Servers
- AlicenseAqualityDmaintenanceMCP server for vision AI — screenshots to code, OCR, error diagnosis, and image analysis via OpenAI-compatible APIs.82MIT
- AlicenseNot gradedqualityCmaintenanceMulti-backend AI vision for MCP agents. Analyze images, screenshots, and documents using local Ollama models or cloud APIs like OpenAI, Google Gemini, and OpenRouter.MIT
- AlicenseNot gradedqualityBmaintenanceLocal vision-capable MCP server that lets AI agents describe screenshots, UI, charts, and photos via vision and OCR tools, with support for multiple providers and automatic fallback.5MIT
- AlicenseAqualityBmaintenanceProvides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.36 npmMIT