Skip to main content
Glama
Sign In
sgs844575

image-understand-mcp

by sgs844575
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

image-understand-mcp

让纯文本模型也拥有图片理解能力的 MCP 服务。

原理

纯文本大模型本身无法处理图片，但可以调用 MCP 工具。本服务作为一个 "视觉能力代理"：

纯文本模型（Claude / GPT / 通义千问 等，无视觉能力）
      │  调用 MCP 工具 understand_image(image, prompt)
      ▼
image-understand-mcp（本服务）
      │  读取本地文件 / 下载 URL / 解析 base64 → data URI
      │  转发给你在配置中指定的视觉模型（OpenAI 兼容接口）
      ▼
视觉模型（GPT-4o / Qwen-VL / GLM-4V / Kimi-VL 等）
      │  返回图片的文字描述
      ▼
纯文本模型继续基于文字描述进行推理、回答

你只需要在 MCP 配置里填一个"看得懂图"的模型的 API 信息，任何纯文本模型 接入这个 MCP 后，就能通过调用工具"看图说话"。

Related MCP server: image_mcp

功能

  • understand_image：分析一张图片，返回详细文字描述或针对指定问题的回答。

    • image 支持三种输入：本地文件路径、HTTP(S) URL、data URI / 裸 base64。

    • prompt 可选，自定义想让视觉模型回答的问题（如"提取图中所有文字"）。

    • provider 可选，当配置了多个视觉模型时可指定使用哪一个。

  • list_vision_providers：列出当前配置的所有视觉模型及默认使用的模型。

安装

npm install
npm run build

配置

所有配置通过环境变量传入（在 MCP 客户端的 env 字段中设置）。

方式一：单个视觉模型（最简单）

{
  "mcpServers": {
    "image-understand": {
      "command": "node",
      "args": ["/path/to/image-understand-mcp/dist/index.js"],
      "env": {
        "VISION_BASE_URL": "https://api.openai.com/v1",
        "VISION_API_KEY": "sk-xxxxxxxx",
        "VISION_MODEL": "gpt-4o"
      }
    }
  }
}

适用于任何 OpenAI 兼容协议的视觉模型服务，例如：

服务商

VISION_BASE_URL 示例

VISION_MODEL 示例

OpenAI

https://api.openai.com/v1

gpt-4o

阿里通义千问

https://dashscope.aliyuncs.com/compatible-mode/v1

qwen-vl-plus

智谱 AI

https://open.bigmodel.cn/api/paas/v4

glm-4v-flash

月之暗面 Kimi

https://api.moonshot.cn/v1

moonshot-v1-8k-vision-preview

硅基流动 SiliconFlow

https://api.siliconflow.cn/v1

Qwen/Qwen2-VL-72B-Instruct

本地 Ollama

http://localhost:11434/v1

llava

方式二：配置多个视觉模型，供调用方按需选择

{
  "mcpServers": {
    "image-understand": {
      "command": "node",
      "args": ["/path/to/image-understand-mcp/dist/index.js"],
      "env": {
        "VISION_PROVIDERS": "[{\"name\":\"gpt4o\",\"baseUrl\":\"https://api.openai.com/v1\",\"apiKey\":\"sk-xxx\",\"model\":\"gpt-4o\"},{\"name\":\"qwen-vl\",\"baseUrl\":\"https://dashscope.aliyuncs.com/compatible-mode/v1\",\"apiKey\":\"sk-yyy\",\"model\":\"qwen-vl-plus\"}]",
        "VISION_DEFAULT_PROVIDER": "gpt4o"
      }
    }
  }
}

配置后，纯文本模型调用 understand_image 时可传入 provider: "qwen-vl" 来指定使用哪个视觉模型；不指定则使用 VISION_DEFAULT_PROVIDER（缺省为 数组中的第一个）。

环境变量说明

变量

必填

说明

VISION_BASE_URL

二选一

视觉模型 API 的 base URL（不含 /chat/completions）

VISION_API_KEY

二选一

API Key

VISION_MODEL

二选一

模型名称

VISION_TIMEOUT_MS

否

单次请求超时时间（毫秒），默认 60000

VISION_PROVIDERS

二选一

JSON 数组，配置多个 provider（见上方示例）

VISION_DEFAULT_PROVIDER

否

多 provider 时的默认 provider 名称

VISION_PROVIDERS 与 VISION_BASE_URL/VISION_API_KEY/VISION_MODEL 二选一； 若同时设置，优先使用 VISION_PROVIDERS。

本地开发

npm run dev          # 用 tsx 直接运行 src/index.ts
npm run build         # 编译到 dist/
npm run inspector      # 用 MCP Inspector 交互式调试（需先 build）
node scripts/smoke-test.mjs   # 端到端冒烟测试（mock 视觉模型 API，无需真实 API Key）

常见问题

Q: 图片必须是什么格式？ A: 支持常见格式（PNG/JPEG/WebP/GIF/BMP），本地路径按扩展名或文件头判断类型， URL 优先用响应头 Content-Type。

Q: 报错 "未找到视觉模型配置"？ A: 检查是否正确设置了 VISION_BASE_URL/VISION_API_KEY/VISION_MODEL， 或 VISION_PROVIDERS。

Q: 视觉模型返回错误？ A: 工具会把视觉模型 API 返回的 HTTP 状态码和错误信息透传给调用方模型， 便于定位是 Key 无效、模型名不对，还是配额不足。

Available Tools

2 tools
list_vision_providers列出可用视觉模型A

列出当前 MCP 服务配置的所有视觉模型 provider 及其名称、模型标识，以及默认使用哪一个。

NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries the full burden. It does disclose the payload shape (provider names, model identifiers, the default), which matters because there is no output schema. It does not explicitly state that the call is side-effect-free, but a configuration listing carries no meaningful ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the action and resource, then the returned fields. Every clause earns its place, and there is no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-argument lister with no output schema, the description covers what the tool is and what comes back, which is adequate. It could go one step further by saying that the returned provider or model identifiers are the values fed to understand_image, which would close the loop with its sibling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters and schema coverage is 100%, so the baseline of 4 applies. There is nothing for the description to disambiguate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb (列出/list) with a specific resource (视觉模型 provider), and it also states the key fields returned: 名称, 模型标识, and 默认使用哪一个. It is clearly distinguishable from the sibling understand_image, which analyzes images, while this tool only enumerates configuration.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The use case (inspecting which vision providers are configured) is strongly implied, and for a zero-parameter listing tool that is largely sufficient. However, it never states when an agent should call this instead of, or before, understand_image, nor does it note any prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

understand_image图片理解A

分析一张图片并返回详细的文字描述。用于让不具备视觉能力的纯文本模型间接'看懂'图片：传入本地文件路径、图片 URL 或 base64 编码数据，本工具会调用配置好的视觉模型完成识别，并返回自然语言描述文本。适合需要理解截图、照片、图表、文档扫描件等内容的场景。

NameRequiredDescriptionDefault
imageYes图片来源，支持三种形式：1) 本地文件绝对/相对路径，如 C:\a.png 或 ./a.png；2) HTTP(S) 图片 URL；3) data URI 或裸 base64 编码字符串。
promptNo想让视觉模型回答的具体问题或分析要求，例如"提取图中的所有文字"" 、判断图中有几个人"。不填则使用默认提示词，返回图片的通用详细描述。
mimeTypeNo仅当 image 为裸 base64 字符串且无法从内容判断格式时使用，例如 image/png、image/jpeg。
providerNo选择使用哪个视觉模型 provider，可选：default（默认）。不填则使用默认 provider。

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden. It usefully discloses the delegation model (an external configured vision model performs recognition and returns natural-language text) and the default-prompt fallback, but says nothing about latency, cost, auth/configuration requirements, or failure modes when the model or image source is unavailable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and return value, then the mechanism, then suitable scenarios — a logical order with no filler. Slightly more verbose than necessary since the mechanism sentence partially duplicates the purpose sentence.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a free-form-text-returning tool with no output schema, the description adequately covers what comes back. But the provider parameter's enum contains only 'default', and the description merely echoes that without explaining how providers get configured or pointing at the sibling list_vision_providers, leaving a real gap for an agent trying to reason about model selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all four parameters are already documented in the schema, which sets the baseline at 3. The description restates the accepted image forms and the default-prompt behavior but adds no syntax or precedence rules (e.g., what happens when both image and mimeType conflict) beyond what the schema already says.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource ('分析一张图片并返回详细的文字描述') and explains the underlying mechanism — delegating to a configured vision model so text-only models can indirectly perceive images. This is much more informative than the title '图片理解', though it never names or distinguishes itself from the sibling list_vision_providers.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear usage context: appropriate for screenshots, photos, charts, and document scans, with three alternative input forms spelled out. However there are no exclusions, no prerequisites, and no pointer to list_vision_providers for discovering or configuring the provider, so the routing cost is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updatesv1.0.0
    • First observedlist_vision_providers
    • First observedunderstand_image
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.9/5.0

Scored across 2 tools

Disambiguation5/5

The two tools have clearly distinct purposes: understand_image performs image analysis, while list_vision_providers only lists configured providers. There is no overlap in function or input, so an agent can easily select the correct tool.

Naming Consistency5/5

Both names use snake_case and follow a verb_noun pattern: understand_image and list_vision_providers. The convention is consistent and predictable across the entire set.

Tool Count3/5

With only 2 tools, the set is borderline thin; one is a core operation and the other is auxiliary. While it matches the narrow purpose, it falls below the typical 3–15 range for a well-scoped server.

Completeness4/5

The core operation of understanding images and a helper to list providers are present, covering the main workflow. A minor gap exists: understand_image uses the configured default model with no parameter to select a specific provider per call, despite list_vision_providers exposing options.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers