Skip to main content
Glama

dsh-vision

支持 OpenAI 协议的 vision MCP 服务器:为 dsh 及其他 harness 提供视觉理解工具。图片/视频/文档理解通过任意 OpenAI 兼容的 chat-completions 端点完成,默认指向智谱 GLM 视觉模型。

  • 工具:vision_analyze(图片/图表问答)、vision_ocr(文字提取)、vision_video(视频理解)、vision_document(文档问答)

  • 输入支持:http(s) URL、本地文件路径、file:// URI、data:...;base64,... URI

  • 传输:stdio(harness 默认)与 streamable-http(远程/Web 场景)

  • 后端:OpenAI 协议 chat-completions;图片走标准 image_url,视频/文档/思考链为智谱扩展(见下方兼容性说明)

安装

git clone <repo> dsh-vision
cd dsh-vision
uv sync                 # 建 .venv 并安装依赖

Related MCP server: glm-vision-mcp

配置(环境变量)

变量

默认值

说明

VISION_API_KEY

API Key(必填)。回退顺序:VISION_API_KEYOPENAI_API_KEYZHIPU_API_KEY

VISION_BASE_URL

https://open.bigmodel.cn/api/paas/v4

OpenAI 兼容端点

VISION_MODEL

glm-4v-plus

默认模型(图片/视频可用 glm-4v-plus;文档与 thinking 需 glm-4.6v+)

VISION_TIMEOUT

300

请求超时(秒)

VISION_MAX_IMAGE_BYTES / VISION_MAX_IMAGES

20 MiB / 4

本地图片大小上限 / 单次图片数量

VISION_MAX_VIDEO_BYTES / VISION_MAX_VIDEOS

50 MiB / 1

本地视频大小上限 / 单次视频数量

VISION_MAX_FILE_BYTES / VISION_MAX_FILES

20 MiB / 4

本地文档大小上限 / 单次文档数量

VISION_DOC_TOOL_TYPE

lite

智谱文档解析工具:lite(免费)expert(PDF 表格/公式更准)prime(复杂排版)

VISION_MAX_DOC_CHARS

500000

解析后的文档文本喂给模型的最大字符数

VISION_TRANSPORT / VISION_HOST / VISION_PORT

stdio / 127.0.0.1 / 8000

streamable-http 模式下的监听参数

运行

# stdio(harness 用)
$env:VISION_API_KEY = "<你的智谱 API Key>"
uv run dsh-vision                          # 或 .venv\Scripts\dsh-vision.exe

# streamable-http
uv run dsh-vision --transport streamable-http --host 127.0.0.1 --port 8000

工具用法

vision_analyze — 图片/图表问答

参数

类型

说明

images

list[str]

图片引用:URL / 本地路径 / data URI(照片、截图、UI、图表、曲线图、原理图均可)

prompt

str

给视觉模型的指令或问题,越具体越好

model

str?

可选模型覆盖(如 glm-4.6v

max_tokens

int?

可选回复长度上限

thinking

bool

开启推理链(智谱 glm-4.6v+,复杂图表/文档推荐开启)

use_search

bool

搜索增强(仅智谱端点):① 提取图中可查证的关键信息与厂商标志/型号 → ② 多路搜索(含厂商官网定向 datasheet 查询,内置 TI/ADI/ST/NXP 等厂商映射)→ ③ 结合证据作答并逐项交叉比对,矛盾点标注来源。更准但慢(约 3-5 次 API 调用),适合冷门/专业内容

vision_ocr — 文字提取

参数

类型

说明

images

list[str]

vision_analyze

vision_video — 视频理解

参数

类型

说明

videos

list[str]

视频引用:URL / 本地路径(mp4/mov/avi/mkv/webm 等)/ data URI

prompt

str

问题或指令,如"总结这个视频"、"第几秒出现人物?"

model / max_tokens / thinking

vision_analyze

vision_document — 文档问答

参数

类型

说明

files

list[str]

文档引用:URL / 本地路径 / data URI(pdf/doc/docx/xls/xlsx/ppt/pptx/txt/md/csv)

prompt

str

问题或指令,如"总结要点"、"第 2 页表格的最大值是多少?"

model / max_tokens / thinking

vision_analyze(文档建议 glm-4.6v

智谱端点的文档走文件解析服务:本地/URL 文档先经解析器(lite 免费 / expert / prime,见配置)提取文本,再把文本与问题一起交给模型回答;扫描版 PDF 建议把 VISION_DOC_TOOL_TYPE 设为 expert(带 OCR)。其他 OpenAI 兼容端点使用通用 file_url 内容块直传。

失败时工具返回 [vision error] ... 文本而不是让 MCP 会话报错,方便模型自行修正重试。

对元件照片(PCB/IC 实物图)这类冷门专业内容,use_search 的价值最明显:

{
  "images": ["OPA1612A-IC.webp"],
  "prompt": "图中是什么芯片?请给出完整型号和关键参数,并说明依据。",
  "use_search": true
}

实测(OPA1612A 音频运放,v0.4.0):厂商定向查询与交叉比对生效——模型主动发现并正确裁决了噪声/供电/THD+N 三处来源矛盾(采信 TI 官方值 1.1nV/√Hz 等);但通道数仍被一处错误搜索摘要带偏(说成单通道)。结论:搜索增强显著提升参数查证能力,但搜索摘要本身可能含幻觉,专业场景请以官方 datasheet 为准(后续改进见 TODO.md)。

兼容性说明

  • 已实测:智谱 GLM —— glm-4v-plus(图片/视频)、glm-4.6v(文档、复杂图表、thinking),均用真实图片、视频和数学建模竞赛 PDF 验证过。

  • ⚠️ 尚未用其他供应商测试:OpenAI、DeepSeek、Qwen/DashScope、本地 vLLM 等只是按 OpenAI 协议实现,没有实测。其中:

    • 图片理解(vision_analyze/vision_ocr)走标准 image_url,兼容性预期较好;

    • video_urlfile_url 内容块、thinking 参数与 use_search 搜索增强都是智谱对 OpenAI 协议的扩展,其他供应商可能不支持或格式不同,使用前请先验证;

    • 智谱端点下 vision_document 使用智谱文件解析服务(非 OpenAI 协议),仅适用于 bigmodel.cn 的 base_url。

  • 欢迎在其他供应商上测试后反馈结果(issue/PR)。

接入 dsh

在 harness 的 MCP 配置中注册 stdio 服务器:

command: .venv\Scripts\dsh-vision.exe   # 或 uv run --directory <项目路径> dsh-vision
env:
  VISION_API_KEY: "<你的智谱 API Key>"

(具体写入位置按 dsh-mcp-install 技能 / harness 文档执行。)

开发

uv run pytest            # 单元测试 + stdio 端到端冒烟
uv build                 # 打包 wheel

端到端脚本(需要智谱 API Key):

uv run python scripts/e2e_zhipu.py --mcp          # 图片链路
uv run python scripts/e2e_media.py                # 视频/文档/图表思考链路
uv run python scripts/e2e_doc.py <文件> <问题>     # 文档问答
uv run python scripts/e2e_search.py               # 搜索增强 vs 普通对比

测试智谱视觉模型

  1. open.bigmodel.cn 创建 API Key

  2. 设置 VISION_API_KEY 后启动服务器

  3. 调用工具,例如:

{"images": ["https://example.com/photo.png"], "prompt": "描述图片内容"}
{"videos": ["C:\\videos\\clip.mp4"], "prompt": "总结这个视频"}
{"files": ["https://example.com/report.pdf"], "prompt": "总结要点", "model": "glm-4.6v"}

Available Tools

4 tools
vision_analyzeA

Analyze one or more images with a vision model.

Works for photos, screenshots, UI layouts, charts, plots, schematics, etc. For text extraction prefer vision_ocr.

Args: images: Image references. Each item may be an http(s) URL, a local file path, or a data:image/...;base64,... data URI. prompt: The question or instruction for the vision model. Be as specific as possible (e.g. "What does this chart show? List the axes and trends."). model: Optional model name override (e.g. glm-4.6v). Defaults to the server's configured VISION_MODEL. max_tokens: Optional cap on the response length. thinking: Enable the reasoning chain (Zhipu glm-4.6v+; ignored by other providers that don't support it). Useful for complex charts/docs.

Returns: The model's text answer.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
imagesYes
promptYes
thinkingNo
max_tokensNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It reveals some behavioral traits like model override defaults and provider-specific handling of the 'thinking' parameter. However, it does not mention side effects, rate limits, authentication needs, or failure modes, which are typical transparency concerns for a tool without annotation support.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with a one-sentence purpose, followed by a compact use-case line and an explicit pointer to OCR. The Args/Returns structure is clean and each sentence provides necessary context without fluff. Despite detailing 5 parameters, it remains appropriately concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, usage, all parameters, and return value. Since an output schema is indicated, return-value detail is sufficient. It could be more complete by mentioning image count/size limits or error behavior, but overall it provides robust context for an AI agent to invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description compensates exceptionally by explaining every parameter in detail: images (URL/path/data URI), prompt (with example), model (default behavior), max_tokens (cap), and thinking (provider compatibility). This adds substantial meaning beyond the raw schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource ('Analyze one or more images with a vision model'), enumerates supported content types, and explicitly distinguishes from a sibling tool ('For text extraction prefer vision_OCR'). This clearly differentiates it from vision_ocr, vision_video, and vision_document.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit alternative for a specific use case (text extraction → vision_OCR) and states general applicability (photos, screenshots, layouts, etc.). However, it does not address when to use vision_video or vision_document, leaving some ambiguity among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_documentA

Ask questions about documents (pdf/doc/docx/xls/xlsx/ppt/pptx/txt/md/csv).

Great for reports, papers, spec sheets, and extracting data from complex tables and charts embedded in documents.

Args: files: Document references. Each item may be an http(s) URL, a local file path, or a data:application/...;base64 data URI. prompt: The question or instruction, e.g. "Summarize the key findings." / "What is the maximum value in the table on page 2?". model: Optional model name override (e.g. glm-4.6v). Defaults to the server's configured VISION_MODEL. max_tokens: Optional cap on the response length. thinking: Enable the reasoning chain (Zhipu glm-4.6v+; ignored by other providers that don't support it). Useful for complex charts/docs.

Returns: The model's text answer.

ParametersJSON Schema
NameRequiredDescriptionDefault
filesYes
modelNo
promptYes
thinkingNo
max_tokensNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It clearly states return value (text answer), explains optional features (thinking enables reasoning chain, notes provider-specific behavior), and notes model default behavior. However, it could be more transparent about document size limitations or auth/access requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured with a one-liner purpose, a usage recommendation sentence, and clearly labeled parameter list. Every sentence adds value, there is no redundancy, and the overall length is appropriate for the tool complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has an output schema (partial burden relief), 5 parameters, and no annotations, the description covers purpose, parameter semantics, usage guidance, and return value completely. It addresses the tool's complexity well, leaving no obvious gaps for safe invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description fully compensates by explaining all 5 parameters: files (URL, path, or data URI), prompt (with example), model (optional override), max_tokens (length cap), and thinking (reasoning chain toggle with provider caveat). This is well beyond what the bare schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool answers questions about various document formats (pdf/doc/docx etc.) and identifies specific use cases like reports, papers, spec sheets, and extracting data from tables/charts. The scope ('ask questions about documents') is specific and distinct from potential sibling tools (e.g., vision_analyze, vision_ocr).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance ('Great for reports, papers, spec sheets...') and provides example prompts. It also implies when alternatives might be needed by listing sibling tools like vision_ocr and vision_video, but does not explicitly say when NOT to use this tool or name specific alternatives for different tasks.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_ocrA

Extract all text from one or more images (OCR via vision model).

Args: images: Image references (http(s) URL, local file path, or data URI).

Returns: The extracted text.

ParametersJSON Schema
NameRequiredDescriptionDefault
imagesYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool extracts all text from images using a vision model, which implies a non-destructive read operation. However, it does not disclose details like whether the model has limitations on image size, supported formats, or rate limits, making the transparency adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the main purpose. The Args and Returns sections are structured clearly. It is reasonably concise, though the Returns line adds minimal value since an output schema exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has only 1 parameter and an output schema, the description covers the key input semantics and purpose. The sibling context adds differentiation. However, it lacks detail on output format (e.g., concatenated or structured) and potential limitations, but the output schema likely covers return structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds some meaning by explaining images are 'Image references (http(s) URL, local file path, or data URI)', which clarifies the type beyond the schema's simple 'string' specification. With schema description coverage at 0% and only 1 parameter, the description compensates partially but could provide more detail like format expectations or size limits.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool extracts all text from images via a vision model. The verb 'Extract' and resource 'images' with OCR via vision model provides a specific purpose, and it distinguishes itself from sibling tools like vision_analyze (likely for analysis not OCR) and vision_video (for video).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description lists accepted image types (URL, path, or data URI) which gives context for when to use, but it does not explicitly say when not to use this tool versus alternatives like vision_analyze or vision_document. The guidelines are implied but lack explicit exclusions or comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_videoA

Understand one or more videos with a vision model.

Args: videos: Video references. Each item may be an http(s) URL, a local file path (mp4/mov/avi/mkv/webm/...), or a data:video/...;base64 data URI. prompt: The question or instruction, e.g. "Summarize what happens in this video." / "At which second does the person enter the frame?". For temporal questions be explicit about time points. model: Optional model name override (e.g. glm-4.6v). Defaults to the server's configured VISION_MODEL. max_tokens: Optional cap on the response length. thinking: Enable the reasoning chain (Zhipu glm-4.6v+; ignored by other providers that don't support it). Useful for complex charts/docs.

Returns: The model's text answer.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
promptYes
videosYes
thinkingNo
max_tokensNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since no annotations are provided, the description carries the full burden. It discloses key behaviors: default model configuration, optional reasoning chain support, and limits on response length. It also notes that thinking is ignored by providers that don't support it. No contradictions with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with a clear header, parameter descriptions, and return value note. It is concise but provides necessary details. A slight reduction from 5 because the parameter explanations could be more terse without losing clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (5 parameters, 2 required) and the presence of an output schema, the description is largely complete. It explains all parameters, usage tips, and return type. It could briefly mention what happens with the 'thinking' parameter for models that do support it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds significant meaning beyond the input schema. It explains that 'videos' can be URLs, local file paths, or data URIs; clarifies what 'prompt' should contain with temporal examples; specifies optional model override and response length cap; and details the 'thinking' parameter's compatibility. This is particularly valuable given the 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it understands one or more videos with a vision model, which is a specific verb+resource combination. It also distinguishes itself from sibling tools like vision_analyze (likely for images/analysis) and vision_ocr (text extraction) by explicitly focusing on videos and temporal questions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool (understanding videos) and includes tips like 'for temporal questions be explicit about time points.' However, it does not explicitly mention when NOT to use it or direct the agent to alternative sibling tools for non-video use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 4 tool updatesv0.1.0
    • First observedvision_analyze
    • First observedvision_document
    • First observedvision_ocr
    • First observedvision_video

TDQS

A4.2/5.0
Disambiguation5/5

Each tool targets a distinct input type: vision_analyze for images, vision_ocr for text extraction from images, vision_video for videos, and vision_document for documents. The description explicitly advises using vision_ocr over vision_analyze for text extraction, eliminating ambiguity.

Naming Consistency4/5

All tools follow a consistent vision_ prefix and use lowercase snake_case. However, vision_analyze uses a verb while vision_ocr, vision_video, and vision_document use nouns or acronyms, creating a minor inconsistency in part of speech.

Tool Count5/5

With 4 tools, the server is well-scoped for its domain of multimodal vision analysis. Each tool addresses a different input modality (images, OCR, videos, documents), leaving no obvious gaps while avoiding unnecessary bloat.

Completeness5/5

The set covers the core capabilities one would expect from a vision MCP server: general image analysis, OCR, video understanding, and document Q&A. No essential operations are missing for the stated purpose of analyzing visual content.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/smjda0719/dsh-vision'

If you have feedback or need assistance with the MCP directory API, please join our Discord server