Skip to main content
Glama
yanickxia

agent-vision

by yanickxia
OverviewSchemaRelated ServersScoreDiscussions
TypeScript
Hybrid

agent-vision

一个小而直接的视觉 MCP Server + Agent Skill：让纯文本 Agent 通过一个 OpenAI-compatible 视觉模型理解图片和视频。

  • 纯 Node.js / TypeScript，npx 自动下载运行

  • MCP 工具：analyze_image、analyze_video

  • Agent Skills 标准目录：skills/agent-vision/，能力自适应：能看图的 Agent 直接看，纯文本 Agent 直接跑 CLI

  • 图片：本地路径、HTTP(S) URL、Data URI

  • 视频：本地路径或 URL，自动下载 npm 内的 ffmpeg 并均匀抽帧

  • 多 provider 多模型并发竞速：所有 target 同时请求，谁先成功用谁，其余请求 abort

为什么做这个项目

参考项目各有优点，但组合起来偏重：

  • luma-mcp：Node.js、npx 和图片预处理体验好，但主要面向单图。

  • deepseek-vision-mcp：MCP 执行层与 Skill 决策层分离得很清楚，但依赖 Python，且本地视频受 provider 限制。

  • vision-tool：支持视频抽帧，但包含大量 provider 探测、并行 fallback 和安装逻辑，并采用 GPL-3.0。

  • agent-vision-toolkit：Skill 和任务工作流很强，但核心是多组 Python/Shell CLI，不是 MCP 视频服务。

agent-vision 只保留一条链路：MCP/CLI → 图片或视频帧 → 你配置的视觉 API → 文本。 本仓库是独立实现，没有复制 vision-tool 的 GPL 源码。

Related MCP server: VisionPower

要求

  • Node.js 20+

  • 一个支持 /chat/completions 和 image_url 的 OpenAI-compatible 视觉模型

ffmpeg 由 @ffmpeg-installer/ffmpeg 按平台自动安装，无需另行安装。

配置

推荐用配置文件（一次配置，所有宿主共用）：

mkdir -p ~/.config/agent-vision
cat > ~/.config/agent-vision/settings.json <<'EOF'
{
  "providers": [
    {
      "name": "dashscope",
      "baseUrl": "https://dashscope.aliyuncs.com/compatible-mode/v1",
      "apiKey": "sk-...",
      "models": ["qwen-vl-max", "qwen-vl-plus"]
    },
    {
      "name": "openai",
      "baseUrl": "https://api.openai.com/v1",
      "apiKey": "sk-...",
      "models": ["gpt-4o"]
    }
  ]
}
EOF
chmod 600 ~/.config/agent-vision/settings.json

  • 位置：$AGENT_VISION_CONFIG 显式路径，或 $XDG_CONFIG_HOME/agent-vision/settings.json（默认 ~/.config/agent-vision/settings.json）

  • 每个 provider × model 是一个 target；多个 target 时并发请求，第一个成功者胜出，其余请求 abort。注意成本：N 个 target 意味着每次分析最多 N 份上传与推理费用

  • provider 字段：baseUrl（必填）、models（必填非空）、name/apiKey/headers（可选；apiKey 缺省回退环境变量）

  • 顶层可选：maxTokens、timeoutMs、headers、maxImageMb、maxVideoMb、allowPrivateUrls、ffmpegPath

  • 文件定义的字段优先于环境变量；含 key 的文件建议 chmod 600

环境变量作为回退仍然完整支持（无配置文件时）：

设置

兼容回退

AGENT_VISION_API_KEY

VISION_API_KEY → OPENAI_API_KEY

AGENT_VISION_BASE_URL

VISION_BASE_URL → OPENAI_BASE_URL

AGENT_VISION_MODEL

VISION_MODEL → OPENAI_MODEL

可选变量：

变量

默认值

说明

AGENT_VISION_MAX_TOKENS

4096

最大输出 tokens

AGENT_VISION_TIMEOUT_MS

120000

下载和 API 超时

AGENT_VISION_MAX_IMAGE_MB

20

图片大小上限

AGENT_VISION_MAX_VIDEO_MB

200

视频大小上限

AGENT_VISION_HEADERS

{}

额外请求头 JSON

AGENT_VISION_ALLOW_PRIVATE_URLS

false

允许私网图片/视频来源 URL

AGENT_VISION_FFMPEG_PATH

自动发现

指定 ffmpeg 可执行文件；默认先用 PATH，再用 npm 内置版本

MCP 安装

npm（发布后推荐）

{
  "mcpServers": {
    "agent-vision": {
      "command": "npx",
      "args": ["-y", "@yanickxia/agent-vision"]
    }
  }
}

直接从 GitHub 运行

{
  "mcpServers": {
    "agent-vision": {
      "command": "npx",
      "args": ["-y", "github:yanickxia/agent-vision"]
    }
  }
}

无需 env 块：配置从 ~/.config/agent-vision/settings.json 读取（见上文「配置」）。 也可以继续用环境变量，在 env / environment 块里传入。

Claude Desktop、Claude Code、Cursor、Cline 等使用上面的标准 mcpServers 格式。

OpenCode

{
  "mcp": {
    "agent-vision": {
      "type": "local",
      "command": ["npx", "-y", "@yanickxia/agent-vision"],
      "enabled": true
    }
  }
}

Agent Skill

npx skills add yanickxia/agent-vision --skill agent-vision -g -y

也可以复制 skills/agent-vision/ 到 Agent 的 skills 目录。Skill 是能力自适应 的：Agent 自己能看图就直接看；看不了（或输入是视频）就由 Skill 指导 Agent 直接跑 CLI 完成分析。

CLI

npx -y @yanickxia/agent-vision image ./screenshot.png \
  --prompt "读取报错并给出可能原因"

npx -y @yanickxia/agent-vision video ./demo.mp4 \
  --prompt "按时间顺序总结 UI 操作" --frames 8

npx -y @yanickxia/agent-vision doctor

MCP 工具

analyze_image

{
  "source": "/absolute/path/to/image.png",
  "prompt": "这个页面有哪些可用性问题？"
}

支持 JPEG、PNG、WebP、GIF、BMP。GIF 是否能体现动画取决于视觉模型；需要稳定的 时间线分析时请转为视频并使用 analyze_video。

analyze_video

{
  "source": "/absolute/path/to/video.mp4",
  "prompt": "概括操作流程和关键变化",
  "max_frames": 8
}

max_frames 范围 1–16，默认 8。实现会均匀抽取 JPEG 帧，并带时间点一起发给 模型。它不是逐帧转写，快速变化可能被漏掉。

隐私与安全

  • 图片会发送到你配置的视觉 API。

  • 视频不会整体发给模型；远程视频先下载到本地临时目录，然后只发送抽出的 JPEG 帧。

  • 临时目录在成功或失败后都会删除。

  • 用户提供的远程图片/视频 URL 默认拒绝 localhost、私网、链路本地和云元数据地址。

  • stdio 模式不向 stdout 写日志，避免破坏 MCP 协议。

开发

npm install
npm run typecheck
npm test
npm pack --dry-run

License

MIT。第三方依赖保留各自许可证，见 NOTICE。

Available Tools

2 tools
analyze_imageAnalyze imageA

Analyze an image from a local path, HTTP(S) URL, or base64 data URI. Use this for screenshots, UI, OCR, diagrams, photos, and visual debugging.

ParametersJSON Schema
NameRequiredDescriptionDefault
promptNoThe user's specific question about the image
sourceYesLocal path, HTTP(S) URL, or data:image/... URI

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of disclosing behavioral traits. It specifies input formats but does not explain what the tool returns (e.g., an answer to the prompt, a description, or analysis results) or any side effects, privacy implications, or limitations. This is a significant gap for a tool that sends image data to an external service.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, consisting of two sentences that immediately state the purpose and then list use cases. It avoids unnecessary repetition and is well-structured, with the action and input types front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This tool has no output schema and no annotations, so the description should compensate by explaining what the tool returns. It fails to mention the prompt parameter or that users can ask specific questions about the image, making the tool's behavior incomplete for an agent deciding how to invoke it. The description covers the input side well but leaves the output side undefined.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already covers both parameters (source and prompt) with clear descriptions, and schema coverage is 100%. The tool description adds context about use cases but does not add meaning beyond the schema for the individual parameters, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Analyze') and resource ('an image'), and enumerates concrete use cases (screenshots, UI, OCR, diagrams, photos, visual debugging). This distinguishes it from its sibling analyze_video, which focuses on video.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage context by listing supported input types (local path, HTTP(S) URL, base64 data URI) and concrete scenarios where the tool should be used. However, it does not explicitly state when not to use it or mention the alternative analyze_video for video content, though this is implied by the sibling name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyze_videoAnalyze videoA

Analyze a local or remote video by extracting evenly spaced keyframes. Use this for screen recordings, UI flows, demos, and event summaries.

ParametersJSON Schema
NameRequiredDescriptionDefault
promptNoThe user's specific question about the video
sourceYesLocal path or HTTP(S) URL
max_framesNoNumber of frames to sample; default 8

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It transparently reveals the keyframe-extraction mechanism and source types (local/remote). It does not mention output format or potential limitations (e.g., loss of temporal context between frames), which prevents a top score, but the core behavior is well outlined.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is exceptionally concise: two sentences, no fluff, with the primary purpose front-loaded and supporting usage guidance in the second sentence. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no output schema, so the description should clarify the return value. While the description explains the mechanism (extracting keyframes) and gives usage examples, it does not state what the analysis produces (e.g., text summary, labeled frames, etc.), leaving a notable gap for an AI agent selecting and invoking the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides detailed descriptions for all three parameters (source, prompt, max_frames), achieving 100% schema coverage. The description adds no extra parameter-level meaning beyond what the schema already states, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: 'Analyze a local or remote video by extracting evenly spaced keyframes.' This specific verb+resource+method distinguishes it from typical image tools. It also lists concrete use cases like 'screen recordings, UI flows, demos, and event summaries,' further clarifying its video-centric purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Use this for screen recordings, UI flows, demos, and event summaries' offers clear, context-rich guidance on when to use the tool. However, it does not explicitly mention when not to use it or compare it with the sibling tool 'analyze_image,' so it stops short of full alternative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.1/5.0
Disambiguation5/5

The two tools are perfectly distinct: analyze_image handles static images, while analyze_video handles moving media via keyframes. There is no overlap in input types or use cases, so an agent can easily choose the correct tool.

Naming Consistency5/5

Both tools follow the exact same verb_noun pattern: analyze_image and analyze_video. This consistent naming makes the tool set predictable and easy to navigate.

Tool Count4/5

With only two tools, the server is minimal but well-scoped for its stated purpose of visual media analysis. While it's on the low end, the narrow domain justifies the count, and each tool covers a major media type.

Completeness5/5

The server covers the two essential types of visual input—images and videos. Both tools are generic enough to handle a wide range of analysis tasks (screenshots, OCR, UI flows, etc.), leaving no obvious gaps in the covered domain.

Maintenance

ActivityMaintained
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/yanickxia/agent-vision'

If you have feedback or need assistance with the MCP directory API, please join our Discord server