VisionToolMCP
VisionToolMCP bridges visual capabilities to text-only agents by sending images (local file paths, base64 data, or public URLs) to multimodal models and returning text and structured results.
Core capabilities:
Describe an Image (
describe_image) — Generate a detailed description of an image, with optional focus or instructions.Extract Text via OCR (
ocr_image) — Extract visible text from an image, with optional language hints and layout preservation.Answer Questions About an Image (
answer_about_image) — Ask a specific question and receive a visually-grounded answer.Compare Two Images (
compare_images) — Analyze and summarize differences between two images, with optional focus instructions.
Additional features:
Supports multiple AI backends: Anthropic, OpenAI, and Gemini
Returns both human-readable text and structured JSON responses
Includes a caller model whitelist, proxy fallback, retry logic, and security controls (path restrictions, URL allowlists, private IP blocking)
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@VisionToolMCPdescribe the image at /screenshots/error.png"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
VisionToolMCP
VisionToolMCP 是一个 MCP 服务器,为纯文本 Agent 提供视觉能力桥梁。它接受图像文件、base64 图像数据或图像 URL,将它们发送到多模态模型,并通过 MCP 返回文本内容和 structuredContent 结构化结果。
功能特性
📤 图像上传 - 通过 base64 上传图像,获得
imageId可重复使用,30 分钟自动清理🔍 图像描述 - 描述图像内容,支持可选的聚焦/指令引导
📝 文字识别 (OCR) - 从图像中提取可见文本
❓ 图像问答 - 回答关于单张图像的特定问题
🆚 图像对比 - 比较两张图像并总结相关差异
Related MCP server: image-viewer-mcp
环境要求
Node.js 20+
设置以下任一 API 密钥(环境变量):
VISIONTOOL_API_KEY、ANTHROPIC_API_KEY、OPENAI_API_KEY或GEMINI_API_KEY
安装
npm install
npm run build运行
VISIONTOOL_API_FORMAT=gemini VISIONTOOL_API_KEY=你的密钥 npm run devMCP 客户端配置
所有配置统一放在项目根目录的 .env 文件中(参考 .env.example 复制为 .env 并编辑)。服务器启动时从项目根加载 .env 且以它为唯一来源(override: true),MCP 客户端的 env 块不需要也不起作用——避免客户端缓存旧配置导致行为漂移。客户端配置只需指向编译后的服务器:
{
"mcpServers": {
"visiontool": {
"command": "node",
"args": ["X:/MCP/VisionToolMCP/dist/index.js"]
}
}
}配置选项
支持以下环境变量配置:
环境变量 | 说明 | 默认值 |
| API 格式: |
|
| 统一 API 密钥,也可使用提供商特定的密钥(如 | - |
| 使用的模型 |
|
| API 基础 URL | - |
| 请求超时(毫秒) |
|
| 本地/base64 图像最大大小 |
|
| 429/5xx 等临时 API 故障的重试次数 |
|
| 指数退避重试的基础延迟 |
|
| 限制本地图像路径必须位于这些根目录下;多个目录用系统路径分隔符分隔 | 不限制 |
| 设为 |
|
| 限制图像 URL host,逗号分隔;支持 | 不限制 |
| 允许 localhost/private IP 图像 URL 被发送给上游视觉模型 |
|
| 网络错误后重试使用的代理 URL |
|
| 设为 |
|
| 调用者模型名黑名单,用逗号分隔;设置后 | 空(默认放行所有调用者) |
| 上传图像的自动过期时间(毫秒) |
|
| 设为 |
|
| 显式指定 opencode 数据库路径;设置后同样启用 | - |
调用者护栏
此 MCP 服务器默认对任何调用者开放,不要求 _caller_model。理由:是否需要视觉能力是上下文判断而非身份判断--同一个多模态模型,能直接看到图片时不该调用,拿到 [Unsupported Image] 时就该调用。靠模型名白名单拦截会误伤正当用户(比如 Claude 因 harness 传输问题看不到图、却需要求助),且白名单本身可被调用方伪造,并非硬保证。
护栏放在工具描述里:每个视觉工具的描述都写明"仅当你无法直接看到图片时调用,能直接看到图的多模态模型请勿调用",由调用方自判。
如果需要硬保证某个模型族永不路由到本服务器(例如防止某个多模态模型误调用浪费额度),可设置可选黑名单:
# 永不允许 claude / gpt 系列模型调用
VISIONTOOL_BLOCK_CALLER_SUBSTRINGS=claude,gpt设置黑名单后:
_caller_model变为必填(否则拒绝,因为无法判断是否在黑名单内)匹配方式为子串包含:只要模型名(含
provider/前缀部分)中出现任一关键字即被挡。例如gpt可挡gpt-4o、opencode/gpt-5.5、azure-gpt-5
完整配置示例请参考 .env.example 文件。
opencode 专属:opencode_pasted_image
这是一个仅 opencode 适用的工具,默认不注册。当 MCP 服务运行在 opencode 会话内时,设置 VISIONTOOL_ENABLE_OPENCODE=1(或 VISIONTOOL_OPENCODE_DB=<opencode.db 路径>)即可启用。
为什么需要它:在 opencode 中粘贴的图片以 base64 data URL 内联存储在 opencode.db 的 part 表里,不落盘。纯文本调用模型(如 GLM/DeepSeek)接不到图片附件,会报 Cannot read "image.png" (this model does not support image input),图片被丢弃。该工具直接从数据库里把"当前会话最新一张粘贴图"取出来,解码落盘并注册为上传,返回 imageId/path 供其它视觉工具使用。
会话识别:用"最新一条 message 所属的 session"作为当前会话(比 session.time_updated 更可靠,后者会被 UI 选中事件刷新)。只在当前会话内取最新图片,不会跨会话兜底——当前会话没有粘贴图时会直接报错,避免拿错别的对话的图。
用法:
{
"name": "opencode_pasted_image",
"arguments": {}
}响应:
{
"tool": "opencode_pasted_image",
"imageId": "6b49d277-c27d-434c-8e52-8601c8f5a1fb",
"path": "C:\\...\\visiontool-mcp-uploads\\image.png",
"bytes": 356351,
"mediaType": "image/png",
"filename": "image.png",
"sessionId": "ses_xxx",
"timeCreated": 1785249912775,
"expiresAt": "2026-07-28T15:31:08.235Z"
}然后用 imageId 调用其它工具:
{
"name": "describe_image",
"arguments": {
"image": { "imageId": "6b49d277-c27d-434c-8e52-8601c8f5a1fb" }
}
}依赖 Node 22.5+ 内置的 node:sqlite(运行时动态导入,不可用时该工具会清晰报错,不影响其它工具)。其它 agent 平台(无 opencode 数据库)请勿启用。
工具调用示例
所有工具默认无需 _caller_model(仅在设置了 VISIONTOOL_BLOCK_CALLER_SUBSTRINGS 黑名单时才必填);reasoningEffort 可选但默认 medium(可填 low / medium / high / xhigh)。
图像上传(推荐)
对于对话中的截图或 Agent 自己生成的图像,先上传获得 imageId 可多次使用,30 分钟后自动清理:
{
"name": "upload_image",
"arguments": {
"base64": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==",
"mediaType": "image/png",
"filename": "screenshot"
}
}响应:
{
"tool": "upload_image",
"imageId": "550e8400-e29b-41d4-a716-446655440000",
"path": "/tmp/visiontool-mcp-uploads/screenshot.png",
"bytes": 1234,
"expiresAt": "2024-06-25T03:30:00.000Z"
}然后用 imageId 调用其他工具:
{
"name": "describe_image",
"arguments": {
"image": {
"imageId": "550e8400-e29b-41d4-a716-446655440000"
}
}
}其他输入方式
也支持 path(本地文件)、base64(直接传数据)或 url(公开 URL):
{
"name": "describe_image",
"arguments": {
"image": {
"path": "X:/screenshots/current.png"
},
"detail": "medium",
"maxTokens": 1024,
"model": "gpt-5.6-terra",
"reasoningEffort": "medium"
}
}所有视觉工具(describe_image / ocr_image / answer_about_image / compare_images)都支持 reasoningEffort 参数(low / medium / high / xhigh,默认 medium),透传给上游模型的思考强度:
OpenAI 格式:映射为
reasoning_effort字段。Anthropic 格式:映射为
thinking.type=enabled,预算按强度取 1024 / 4096 / 8192 / 16384 token(预算必须小于maxTokens才会发送)。Gemini 格式:映射为
generationConfig.reasoningConfig.effort。不传时该字段被省略,使用模型自身的默认推理强度。
注意:各视觉工具默认 maxTokens 为 2048(ocr_image 2048、compare_images 1536),保证 Anthropic 格式下默认的 reasoningEffort: medium 预算(1536 token)始终有效;如果手动把 maxTokens 调低到 1365 以下,Anthropic 的 thinking 预算会因不满足 >= 1024 而被跳过。
工具响应会同时包含可读文本和结构化对象。结构化对象字段为:
{
"tool": "describe_image",
"model": "gemini-2.5-flash",
"apiFormat": "gemini",
"text": "model response text",
"images": [
{
"source": "path",
"mediaType": "image/png",
"path": "X:/screenshots/current.png",
"bytes": 12345
}
]
}图像输入安全边界
imageId- 通过upload_image返回的 ID,30 分钟自动清理(推荐,最安全)path- 绝对或相对本地图像路径base64- 原始 base64 图像数据url- 可公开访问的图像 URL
支持的 MIME 类型:PNG、JPEG、WebP 和 GIF。
注意:path 会读取 MCP 服务器进程可访问的本地文件;url 会把 URL 交给上游视觉模型提供商读取。生产环境建议设置 VISIONTOOL_ALLOWED_IMAGE_ROOTS、VISIONTOOL_ALLOWED_URL_HOSTS,或用 VISIONTOOL_DISABLE_URL_INPUTS=1 禁用 URL 输入。默认会拒绝 localhost/private IP URL;确有需要时才设置 VISIONTOOL_ALLOW_PRIVATE_URLS=1。
网络与代理
遇到可重试的网络错误时,服务器会先直连,再通过代理 fallback 重试。代理默认不启用(不猜测任何本地代理端口,避免把带 API key 的请求发给无关的本机服务);仅在显式设置了 VISIONTOOL_PROXY_URL(或环境中的 HTTPS_PROXY / HTTP_PROXY)时才启用,选择顺序为 VISIONTOOL_PROXY_URL、HTTPS_PROXY、HTTP_PROXY。设置 VISIONTOOL_DISABLE_PROXY_FALLBACK=1 可强制关闭。
开发
npm test
npm run buildAgent 使用说明
此服务器与截图/捕获 MCP 配合使用。先用另一个工具截取屏幕截图,将返回的文件路径传递给 describe_image 或 answer_about_image,然后使用结构化的文本响应来决定下一步操作。
Available Tools
4 toolsanswer_about_imageA
【仅限 GLM/DeepSeek 系列模型调用】Answer a specific question using visual evidence from one image.
| Name | Required | Description | Default |
|---|---|---|---|
| image | Yes | ||
| question | Yes | Question to answer using visual evidence from the image. | |
| detail | No | How much visual detail to request from the vision model. | medium |
| maxTokens | No | ||
| model | No | Optional Claude model override. Defaults to VISIONTOOL_MODEL or claude-opus-4-8. | |
| _caller_model | No | 【限制项】调用方模型名。仅限 GLM / DeepSeek 系列模型调用此工具,其他模型将被拒绝。 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the disclosure burden. It reveals the model restriction (GLM/DeepSeek only) and implies a text output. However, it does not detail error handling, return format, or other behavioral traits like rate limits or auth requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficient with one core sentence and a necessary prefix. It is front-loaded and contains no wasted words. The Chinese prefix could be confusing for non-Chinese agents but is essential for the restriction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, nested object schema, no output schema, no annotations), the description is too brief. It lacks details on output format, error cases, and process expectations. The sibling tools are not cross-referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema coverage is 67% (medium) and the description adds no extra meaning to parameters. It does not explain parameter usage or constraints beyond what is in the schema. For a tool with multiple parameters including nested objects, this is insufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: answering a specific question using visual evidence from one image. It uses a specific verb and resource, and distinguishes itself from sibling tools like compare_images (two images), describe_image (general description), and ocr_image (text extraction).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use this tool (for answering specific questions about a single image) vs. siblings. The Chinese prefix explicitly restricts usage to GLM/DeepSeek models. However, it does not explicitly state when not to use it or provide alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
compare_imagesA
【仅限 GLM/DeepSeek 系列模型调用】Compare two images and summarize relevant differences for a text-only agent.
| Name | Required | Description | Default |
|---|---|---|---|
| firstImage | Yes | ||
| secondImage | Yes | ||
| instruction | No | Optional comparison instruction or focus. | |
| detail | No | How much visual detail to request from the vision model. | medium |
| maxTokens | No | ||
| model | No | Optional Claude model override. Defaults to VISIONTOOL_MODEL or claude-opus-4-8. | |
| _caller_model | No | 【限制项】调用方模型名。仅限 GLM / DeepSeek 系列模型调用此工具,其他模型将被拒绝。 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states that it 'summarizes differences' without disclosing behavioral traits like error handling, authentication needs, or effect on system. This is insufficient for a tool with no annotation safety net.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single sentence plus a bracketed restriction, with no wasted words. It is front-loaded with the action and model restriction.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (nested objects, multiple image input methods, no output schema), the description is minimal. It does not explain image input options or output format, but the schema covers input details. Adequate but with gaps for an agent to fully understand without schema inspection.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 57%, but the description text adds no additional meaning to parameters beyond what the schema provides. It does not mention image input options or any parameter details, leaving a gap in understanding for complex nested inputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Compare two images and summarize relevant differences' with a specific verb and resource. It distinguishes from sibling tools (single-image tasks) by explicitly mentioning comparison and two images.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description includes a model restriction ('仅限 GLM/DeepSeek 系列模型调用') which guides when to use, and the context of 'for a text-only agent' implies suitable scenarios. However, it does not explicitly exclude alternatives or mention prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
describe_imageC
【仅限 GLM/DeepSeek 系列模型调用】Describe an image for a text-only agent. Accepts a local path, base64 image data, or URL.
| Name | Required | Description | Default |
|---|---|---|---|
| image | Yes | ||
| instruction | No | Optional extra instruction for what to describe. | |
| focus | No | Optional visual area or topic to focus on. | |
| detail | No | How much visual detail to request from the vision model. | medium |
| maxTokens | No | ||
| model | No | Optional Claude model override. Defaults to VISIONTOOL_MODEL or claude-opus-4-8. | |
| _caller_model | No | 【限制项】调用方模型名。仅限 GLM / DeepSeek 系列模型调用此工具,其他模型将被拒绝。 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must carry behavioral disclosure. It does not mention authorization needs, rate limits, error handling, or output format. Only states basic function.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is concise and front-loaded with essential information (model restriction and function). However, could be better structured with clearer sections.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has 7 parameters with nested objects and no output schema. Description fails to explain return values, error behavior, or usage examples. Insufficient for a complex tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 71% of parameters with descriptions. Description adds limited value beyond schema (e.g., listing input methods). Baseline score due to high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it describes an image for a text-only agent and lists input types (path, base64, URL). Distinguishes from sibling tools like 'compare_images' and 'ocr_image' implicitly by focusing on description, but could be more explicit.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives like 'answer_about_image'. Includes a model restriction (GLM/DeepSeek) but does not explain usage context or edge cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
ocr_imageB
【仅限 GLM/DeepSeek 系列模型调用】Extract visible text from an image with optional language and layout hints.
| Name | Required | Description | Default |
|---|---|---|---|
| image | Yes | ||
| language | No | Optional language hint for visible text. | |
| preserveLayout | No | Preserve rough reading order and layout where possible. | |
| detail | No | How much visual detail to request from the vision model. | medium |
| maxTokens | No | ||
| model | No | Optional Claude model override. Defaults to VISIONTOOL_MODEL or claude-opus-4-8. | |
| _caller_model | No | 【限制项】调用方模型名。仅限 GLM / DeepSeek 系列模型调用此工具,其他模型将被拒绝。 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does not disclose read-only nature, error handling (e.g., no text found), or return format. The model restriction is mentioned but other behavioral traits like resource consumption or side effects are omitted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief (two sentences) and front-loaded with the purpose. The Chinese note is important but adds some clutter; an English-only agent might need parsing. Still, the structure is efficient with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is provided, and the description does not explain what the tool returns (e.g., extracted text, structure). With 7 parameters, details like maxTokens behavior, detail levels, and model override are not mentioned, leaving significant gaps for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 71% (5 of 7 parameters have descriptions). The description adds context for 'language' and 'preserveLayout' as optional hints, but most parameter semantics are already in the schema. The _caller_model parameter's schema description already includes the restriction; description does not add beyond that.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool extracts visible text from an image, which matches the name 'ocr_image'. It mentions optional language and layout hints, distinguishing it from sibling tools that answer questions, compare, or describe images. However, it does not explicitly differentiate from siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a model restriction note ('仅限 GLM/DeepSeek 系列模型调用'), which is a usage guideline. But it lacks explicit when-to-use or when-not-to-use guidance compared to sibling tools (e.g., use for text extraction, not for image interpretation).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
4 tool updates
v0.1.0- First observed
answer_about_image - First observed
compare_images - First observed
describe_image - First observed
ocr_image
TDQS
Scored across 4 tools
Each tool targets a distinct capability: answering specific questions, comparing two images, describing content, and extracting text. There is no overlap in their purposes.
All tools use snake_case with verb-first pattern (answer, compare, describe, ocr). However, 'answer_about_image' uses a preposition while others directly combine verb and noun, a minor inconsistency.
Four tools is appropriate for a focused vision server providing core image understanding capabilities. Not too few or too many.
Covers key image interpretation needs: description, comparison, OCR, and question answering. Missing potential features like object detection or image generation, but the set is reasonably complete for its stated purpose of supporting text-only agents.
Maintenance
Related MCP Connectors
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Create images & video from any MCP agent — 17 models, spend limits, one URL.
Image processing over MCP: compress and split images, generate images and QR codes.
Generate on-brand images from your AI agent: design, edit, and render templates over MCP.
Related MCP Servers
- AlicenseAqualityBmaintenanceEnables non-multimodal models to see images by providing MCP tools for image understanding and OCR, backed by any OpenAI-compatible vision model.2MIT
- FlicenseNot gradedqualityCmaintenanceProvides an MCP tool that analyzes images from local paths, URLs, or data URLs via a vision language model, returning structured descriptions (brief, detailed, summary) so text-only LLMs can understand image content.-
- AlicenseNot gradedqualityCmaintenanceEnables MCP-compatible agents to analyze images via NVIDIA NIM vision models, supporting file paths, URLs, or base64 input to return actionable textual descriptions.7 npmMIT
- AlicenseAqualityCmaintenanceEnables any MCP-capable agent to perform vision tasks like describing images, answering questions, OCR, and comparing images using supported vision backends.5MIT