mcp-multivision-server
The MCP Multivision Server provides both cloud-based AI vision analysis and local media processing capabilities for images and videos.
Cloud-Based Vision Analysis (requires API key)
Image Analysis (
vision_analyze_image): Describe images, answer questions, perform OCR, interpret charts/UI/diagrams, and diagnose errors — using configurable task presets or custom prompts.Video Understanding (
vision_analyze_video): Send videos directly to cloud multimodal models (e.g., Qwen/DashScope) for temporal scene, object, action, and event analysis.
Local Video Processing (requires ffmpeg, no API key)
Video Metadata (
vision_video_info): Get duration, resolution, frame rate, codec, and stream info viaffprobe.Frame Extraction (
vision_extract_frames): Extract frames as image files by count, keyframe, or time interval.Audio Extraction (
vision_extract_audio): Extract audio tracks as MP3 or WAV.Scene Detection (
vision_scene_detect): Detect scene change timestamps using ffmpeg.
Local Image Processing (no API key needed)
Image Metadata (
vision_image_metadata): Read dimensions, format, color mode, and EXIF data (including GPS) via Pillow.Face Detection (
vision_detect_faces): Detect frontal faces using OpenCV Haar cascades, returning count and bounding boxes.Image Similarity (
vision_compare_images): Compare two images via perceptual hashing and color histograms, returning a 0–1 similarity score.
Server Utilities
Server Status (
vision_get_server_status): Check cloud model configuration, ffmpeg availability, and current settings.
Input formats supported: local absolute paths, file:// URLs, http(s):// URLs, and base64-encoded data (images). Communication via stdio or SSE transport.
Provides image and video analysis capabilities by leveraging OpenAI compatible vision language models, including support for multiple providers such as OpenAI, 通义千问VL, GLM-4V, and 豆包.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@mcp-multivision-server Describe the image at /home/user/photo.jpg"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP Multivision Server
为大模型提供图片与视频分析能力的 MCP 服务器,纯云端:所有视觉理解都交给
OpenAI 兼容的云端视觉大模型完成,视频原生交给模型处理(如 Qwen3.5,通过 DashScope video_url)——
本地不做任何抽帧/转码,镜像轻量、无 ffmpeg / OpenCV 依赖。
设计:视频的抽帧与时序对齐由模型服务端完成。一份
base_url + api_key + model配置即可切换 通义千问3.5 / Qwen-VL / GLM-4V / GPT-4o(GPT-4o 仅图片,不支持video_url)。
功能
图片理解问答:描述、问答、OCR、图表解读、UI 分析、示意图理解、报错诊断(
vision_analyze_image,含任务预设)视频理解(原生):把视频直接交给云端多模态大模型,按时间顺序分析场景/对象/动作/事件(
vision_analyze_video)图片元信息(本地):EXIF/尺寸/格式,Pillow 解析(
vision_image_metadata)支持 stdio 与 SSE 两种传输
输入支持:本地绝对路径 /
file:///http(s):/// base64(data URI);视频推荐传 http(s) URL
Related MCP server: mcp-ffmpeg
快速开始
1. 配置云端视觉模型(OpenAI 兼容)
视频需选支持原生视频的模型:
平台 | MCP_VISION_BASE_URL | 示例 MCP_VISION_MODEL | 视频 |
通义千问3.5 |
|
| ✓ |
通义千问VL |
|
| ✓ |
智谱 GLM-4V |
|
| 视模型 |
OpenAI |
|
| ✗(仅图片) |
video_url是 DashScope 对 OpenAI 协议的扩展,因此原生视频当前主要在通义千问系列可用; OpenAI 官方 GPT-4o 不支持video_url,只能做图片。
2. 本地开发运行
无需 ffmpeg,纯 Python 依赖:
python -m venv .venv && source .venv/bin/activate
pip install -e ".[sse]"
# stdio(本地 MCP 客户端)
mcp-multivision-server
# SSE(远程 MCP 客户端)
MCP_TRANSPORT=sse MCP_PORT=8093 mcp-multivision-server部署
cp .env.example .env # 填入 MCP_VISION_BASE_URL / API_KEY / MODEL
docker compose up -d镜像基于 python:3.11-slim,无 ffmpeg / OpenCV / 系统库依赖,体积约 200MB。
推送 v* tag 触发 GitHub Actions:原生 amd64 + arm64 构建、推送 Harbor、多架构 manifest、GitHub Release。
需配置仓库 secrets HARBOR_USERNAME / HARBOR_PASSWORD。
MCP 客户端配置
SSE:
{ "mcpServers": { "multivision": { "url": "http://<your-server>:8093/sse" } } }stdio:
{
"mcpServers": {
"multivision": {
"command": "mcp-multivision-server",
"env": {
"MCP_VISION_BASE_URL": "https://dashscope.aliyuncs.com/compatible-mode/v1",
"MCP_VISION_API_KEY": "your_api_key",
"MCP_VISION_MODEL": "qwen3.5-plus"
}
}
}
}环境变量
变量 | 默认值 | 说明 |
|
|
|
|
| SSE 监听地址 |
|
| provider 名(当前支持 openai 兼容) |
| — | 视觉模型 API 基址 |
| — | 视觉模型 API Key |
| — | 模型名(视频需支持原生视频,如 qwen3.5-plus) |
|
| 生成上限 |
|
| 采样温度 |
|
| 请求超时(秒),视频较慢 |
|
| 失败重试次数 |
|
| 单图 base64 字节上限(20MB) |
|
| 本地视频转 base64 上限(100MB),更大请传 URL |
|
| 允许的图片格式 |
|
| 是否缓存分析结果 |
|
| 缓存目录 |
MCP 工具列表
工具 | 说明 | 是否需云端 key |
| 图片描述/问答/OCR/图表/UI/报错(含 preset) | 是 |
| 视频原生交给云端模型理解(video_url) | 是 |
| 尺寸/格式/EXIF/GPS(本地 Pillow) | 否 |
| 配置与可用性自检 | 否 |
vision_analyze_image 的 preset 可选:describe / ocr / chart / ui / diagram / error;提供 prompt 时覆盖 preset。
项目结构
03-mcp-multivision-server/
├── Dockerfile / docker-compose.yaml / .env.example
├── pyproject.toml / requirements.txt
├── .github/workflows/build-release.yaml
└── src/mcp_multivision_server/
├── server.py # 入口 + 4 个工具
├── providers/ # 云端 VLM
│ ├── base.py # BaseVisionProvider / VisionResult / ProviderError
│ └── openai_compat.py # OpenAI 兼容 provider(含 video_url)
└── media/
├── inputs.py # 输入解析:图片/视频 → image_url / video_url
└── imageinfo.py # 本地 EXIF/尺寸(Pillow)许可
MIT
Available Tools
4 toolsvision_analyze_imageA
图片理解问答:调用云端视觉大模型对图片进行描述、问答、OCR、图表/UI/报错分析等。可用 prompt 自由提问,或用 preset 选择任务类型。
Inputs: image (required), preset, prompt, max_tokens, +1 more.
vision_analyze_videoA
视频理解:将视频原生交给云端多模态大模型(如 Qwen3.5,通过 DashScope video_url)进行理解——抽帧与时序对齐由模型服务端完成,本地不做任何处理。推荐传视频的 http(s) URL;本地文件会转 base64(受大小限制)。
Inputs: video (required), prompt, max_tokens, temperature.
vision_get_server_statusA
查询服务状态:云端视觉模型是否已配置、当前模型与配置项。
Inputs: no named parameters.
vision_image_metadataA
图片元信息:读取尺寸、格式、颜色模式与 EXIF(含拍摄时间、GPS 若有),本地 Pillow 解析,不调用云端。
Inputs: image (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 4 tools
Each tool targets a distinct function: image analysis, video analysis, server status, and metadata extraction. No overlap in purpose, and descriptions clearly differentiate them.
Tools follow a consistent 'vision_<verb>_<target>' pattern for three tools, but 'vision_image_metadata' lacks a verb, though it remains readable.
Four tools is well-scoped for a vision server, covering analysis, status, and metadata without unnecessary proliferation.
The tool surface covers core operations: image/video analysis, server status, and image metadata. No obvious gaps for the stated purpose.
Maintenance
Related MCP Connectors
Transform video, audio and images, and generate media from prompts. FFmpeg, captions, models.
Multimodal video analysis MCP — transcription, vision, and OCR for any video URL.
LLM chat, text tools, image generation, editing, batch image jobs, and asynchronous video generation
Video analysis AI: transcripts, summaries, visual scenes/shots, clips, answers in natural language.
Related MCP Servers
- AlicenseAqualityDmaintenanceEnables AI agents to analyze images, extract text, compare images, and analyze video through any OpenAI-compatible vision model.4488 npm20MIT
- AlicenseAqualityDmaintenanceEnables local media processing (video/audio) using FFmpeg and FFprobe, allowing frame extraction, audio conversion, and metadata retrieval through natural language.5MIT
- AlicenseNot gradedqualityCmaintenanceProvides local image understanding for text-only LLMs with tools for image analysis, OCR, object detection, and cropping, all processed on-device.2MIT
- AlicenseAqualityDmaintenanceProvides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.317 npmMIT