Skip to main content
Glama
CaoMeiYouRen

vision-augment

by CaoMeiYouRen
OverviewSchemaRelated ServersScoreDiscussions
Python
Hybrid

vision-augment

本地优先的多模态视觉 MCP —— 为无视觉 LLM（DeepSeek、GLM 等）提供可自定义端点的看图 / OCR / 文档解析能力。

  • 简化配置、本地优先：不强制依赖云端；OCR 与文档解析全部本地完成

  • 视觉理解用视觉模型：OpenAI 兼容通道链，按序降级，最后兜底本地 Ollama VL（无 key 即用）

  • 不限制模型渠道：任意 OpenAI 兼容端点，base_url + api_key + model 三元组可配多个

  • uvx 一键分发：uvx vision-augment 直接接入任意 MCP harness

架构

MCP client (Hermes / Claude / OpenCode ...)
   └─ mcp_vision_augment_vision(task_type=reasoning|ocr|document)
        └─ Router ─┬─ reasoning → OpenAI 兼容通道链 + Ollama 兜底
                   ├─ ocr       → RapidOCR（本地）→ PaddleOCR（可选升级）
                   └─ document  → markitdown（本地）→ PaddleOCR-VL / MinerU（可选升级）

所有工具返回统一 JSON envelope：{task_type, tool_used, code, error, result, confidence, metadata}，错误码 0-5。

Related MCP server: vision-bridge-mcp

快速开始

环境要求：Python >= 3.12，uv。

方式一：无 key，本地 Ollama（默认）

ollama pull llava  # 或任一视觉模型（llava/vision/qwen2.5-vl/...）
uvx vision-augment

方式二：自定义 OpenAI 兼容端点（推荐）

uvx vision-augment[ocr,document]   # 按需安装本地 OCR/文档引擎
# 注册到 Hermes config.yaml（其他客户端见下）
mcp_servers:
  vision-augment:
    command: uvx
    args: [vision-augment]
    env:
      VISION_AUGMENT_CHANNELS: '[{"base_url": "https://api.example.com/v1", "api_key": "${API_KEY}", "model": "qwen3.7-plus"}]'

无通道配置时自动探测本地 Ollama VL 模型作为兜底；通道配置优先。

本地 OCR/文档引擎（按需安装）

  • 默认安装（uvx vision-augment）只含视觉理解（云端通道/Ollama）；OCR 与文档解析是本地引擎，按需通过 extras 安装：

    • [ocr]：RapidOCR（ONNX，跨平台轻量）——uvx vision-augment[ocr]

    • [document]：markitdown（docx/pdf/pptx/xlsx/html → Markdown）——uvx vision-augment[document]

  • 引擎未安装时调用对应任务返回 code=4 dependency_missing（错误信息附安装命令）；随时可用 mcp_vision_augment_health 查看引擎可用状态

  • ⚠️ 超时注意：RapidOCR 引擎首次加载（ONNX 模型初始化）较慢，首个 OCR 请求可能触发客户端 MCP 超时——重试一次即可（引擎在 server 进程内缓存，第二次起秒回）；若频繁超时，调大客户端 MCP 工具超时（如 opencode 的 toolTimeout），或改用 HTTP 传输（见方式四）

方式三：从 GitHub 直接安装（未发布到 PyPI 前）

# 最新 master（PEP 508 语法：extras 在 @ 之前）
uvx "vision-augment[ocr,document] @ git+https://github.com/CaoMeiYouRen/vision-augment"

# 锁定 tag / commit
uvx "vision-augment @ git+https://github.com/CaoMeiYouRen/vision-augment@v0.1.0"

# 长期安装到 PATH（等价 pipx）
uv tool install "vision-augment[ocr] @ git+https://github.com/CaoMeiYouRen/vision-augment"

方式四：HTTP 传输（streamable-http）

适合 Docker 部署、远程服务器、多客户端并发场景（stdio 单进程只能服务一个客户端）：

VISION_AUGMENT_TRANSPORT=streamable-http VISION_AUGMENT_PORT=8000 uvx vision-augment

  • 默认绑定 127.0.0.1:8000，MCP 端点 /mcp；SDK 对 localhost 自动启用 DNS rebinding 防护

  • 跨机器访问：设置 VISION_AUGMENT_HOST=0.0.0.0，并自行加反向代理/鉴权（远程暴露是部署方责任）

  • 客户端配置示例（Hermes）：

mcp_servers:
  vision-augment:
    url: http://127.0.0.1:8000/mcp
    transport: streamable-http

注意：不要给 streamable-http 端点发送空 params 的 initialize 探测请求——SDK 会挂起该请求，用合法握手载荷探测。

方式五：Docker 部署（streamable-http）

直接使用 CI 构建发布的多架构镜像（linux/amd64 + linux/arm64，发布到 Docker Hub / ghcr.io / 阿里云三个渠道，tag：latest / 日期 / sha-<短hash>）：

docker compose up -d
# 等价：docker run -d --name vision-augment -p 127.0.0.1:8000:8000 caomeiyouren/vision-augment

  • 自定义镜像源：DOCKER_IMAGE=ghcr.io/caomeiyouren/vision-augment docker compose up -d

  • 通道/密钥等配置通过环境变量或 .env 注入，示例见 docker-compose.yml

  • 本地开发构建：docker build -t vision-augment .（构建上下文直接安装源码，不依赖 PyPI）

与 Hermes 同 compose 网络接入（共享 networks 后走容器名）：

mcp_servers:
  vision-augment:
    url: http://vision-augment:8000/mcp
    transport: streamable-http

注册到其他客户端

env 字段用于注入通道配置与密钥（与 Hermes 示例中的 env 同理）；通道为空时自动探测本地 Ollama 兜底。

// Claude Desktop: claude_desktop_config.json
{
  "mcpServers": {
    "vision-augment": {
      "command": "uvx",
      "args": ["vision-augment"],
      "env": {
        "VISION_AUGMENT_CHANNELS": "[{\"base_url\": \"https://api.example.com/v1\", \"api_key\": \"...\", \"model\": \"qwen3.7-plus\"}]"
      }
    }
  }
}
// OpenCode: opencode.json（--from 指定 extras 可启用本地 OCR/文档引擎）
{
  "mcp": {
    "vision-augment": {
      "type": "local",
      "command": ["uvx", "--from", "vision-augment[ocr,document]", "vision-augment"],
      "enabled": true,
      "env": {
        "VISION_AUGMENT_CHANNELS": "[{\"base_url\": \"https://api.example.com/v1\", \"api_key\": \"...\", \"model\": \"qwen3.7-plus\"}]"
      }
    }
  }
}

注意：修改配置（通道/模型/extras）后需重启客户端会话——MCP server 在启动时加载 env，不重启仍是旧配置。

工具

工具

说明

mcp_vision_augment_vision

入口：task_type（reasoning 看图问答 / ocr 图片文字 / document 文档解析）+ source（路径 / file:// / http(s):// / data:URL）+ task + language

mcp_vision_augment_health

环境探测：通道/Ollama/OCR/文档引擎配置状态（不含密钥），供 agent 反馈缺失配置

mcp_vision_augment_clear_cache

清除本地结果缓存

安装与使用 Skill

仓库根目录的 SKILL.md 符合 Agent Skills 规范，可通过 npx skills 生态一键安装（需仓库已公开）：

# 全局安装到 opencode / hermes-agent
npx skills add CaoMeiYouRen/vision-augment -g -a opencode -a hermes-agent -y

# 或项目级安装（不指定 -g）
npx skills add CaoMeiYouRen/vision-augment

# 查看已安装
npx skills list

Skill 安装后 agent 的工作方式：

  1. 环境探测：优先调用 mcp_vision_augment_health 检查通道/Ollama/引擎状态，自动向你反馈还缺哪些配置及安装命令（如 uvx vision-augment[ocr]）

  2. 任务路由：看图 → reasoning；图片文字 → ocr；文档解析 → document，由 skill 指引 agent 选择

  3. 故障闭环：错误码 0-5 对应的处置路径写在 SKILL.md 中

手动安装：把 SKILL.md 复制到 ~/.config/opencode/skills/vision-augment/（opencode）或 ~/.hermes/skills/（Hermes）等目录即可。

配置（环境变量，均有默认值）

变量

默认

说明

VISION_AUGMENT_CHANNELS

[]

OpenAI 兼容通道 JSON 数组

VISION_AUGMENT_OLLAMA_URL

http://127.0.0.1:11434

本地 Ollama 地址

VISION_AUGMENT_CACHE_TTL_SECONDS

86400

缓存 TTL（0=关闭）

VISION_AUGMENT_CACHE_DIR

~/.cache/vision-augment

缓存目录

VISION_AUGMENT_MAX_INPUT_MB

50

输入大小上限

VISION_AUGMENT_MAX_OUTPUT_CHARS

20000

输出截断上限

VISION_AUGMENT_ALLOW_URLS

false

允许 http(s) 输入（防 SSRF，默认关闭）

VISION_AUGMENT_DEBUG

false

DEBUG 日志

VISION_AUGMENT_TRANSPORT

stdio

传输方式：stdio / streamable-http

VISION_AUGMENT_HOST

127.0.0.1

HTTP 绑定地址

VISION_AUGMENT_PORT

8000

HTTP 端口

完整说明见 docs/design.md。

开发

uv sync            # 安装开发环境（基础依赖）
uv run pytest      # 单元测试（不依赖重型引擎）
uv run ruff check  # 代码检查

安装可选引擎做集成验证：

uv sync --extra ocr --extra document   # RapidOCR + markitdown
# 或全量：uv sync --all-extras（含 PaddleOCR，体积大）

发布（CI 自动）

push 到 master 后，release workflow 由 python-semantic-release 根据 conventional commits 自动版本化（pyproject + __version__ + CHANGELOG + tag + GitHub Release），并通过 Trusted Publisher（OIDC，免 token） 发布到 PyPI。

Trusted Publisher 配置（PyPI → Publishing → Trusted Publishers → Add pending publisher）：

字段

值

PyPI Project Name

vision-augment

Owner

CaoMeiYouRen

Repository name

vision-augment

Workflow name

release.yml

Environment name

留空

首次发布后 uvx vision-augment 即生效。手动发布备选：uv build && uv publish（需 UV_PUBLISH_TOKEN）。

文档

  • 需求文档（含需求评估与未覆盖问题评估）

  • 设计文档（架构、envelope 契约、通道链、缓存、平台支持矩阵）

  • SKILL.md（agent 使用技能）

许可证

MIT © 2026 CaoMeiYouRen

Available Tools

3 tools
mcp_vision_augment_clear_cacheA

清除本地结果缓存（缓存 TTL 上限由 VISION_AUGMENT_CACHE_TTL_SECONDS 控制）。

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses the action (clears local result cache) and notes the TTL upper limit controlled by VISION_AUGMENT_CACHE_TTL_SECONDS, providing some context. However, it does not describe any side effects, permissions required, or the impact of clearing the cache.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that conveys the core purpose and an environmental constraint. It is succinct with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple cache-clearing tool with no parameters, the description is mostly adequate. It covers the action and a relevant configuration detail, but lacks usage context and any note about when this would be appropriate or how it interacts with the sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes no parameters, and schema coverage is 100% with zero properties. The description does not need to explain parameters, and none are present accordingly.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function: '清除本地结果缓存' (clear local result cache). It uses a specific verb and resource, but does not differentiate from sibling tools mcp_vision_augment_vision or mcp_vision_augment_health, which are distinctly different operations.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool or when not to. The description implies a cache-clearing operation but does not mention scenarios or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_vision_augment_healthA

环境探测：检查视觉通道/Ollama/OCR/文档引擎的配置状态（不含密钥）， 用于向用户反馈还缺少哪些配置及对应安装命令。

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It explicitly states that the tool excludes keys ('不含密钥'), which is an important behavioral trait regarding sensitive data. It also implies a read-only, non-destructive nature by calling it a probe/detection, but does not explicitly confirm side-effect-free behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that front-loads the main purpose ('环境探测') and then elaborates on components and outputs. Every clause adds value without redundancy or unnecessary details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no params, no output schema), the description covers essential aspects: what is checked, that keys are excluded, and the kind of feedback provided (missing configurations and install commands). It does not specify the exact output format, but this is a minor gap for a health check tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and the schema is empty (100% coverage). Baseline for no parameters is 4; no additional parameter explanations are necessary since there are none to describe.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb (检查/check) and concrete resources (视觉通道/Ollama/OCR/文档引擎), clearly indicating a health/environment probe. It distinguishes itself from siblings like mcp_vision_augment_vision (which likely performs vision tasks) and clear_cache (cache management) by focusing on configuration status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool: to detect missing configurations and provide installation commands. However, it does not explicitly mention when not to use it or name alternative tools, so it stops short of full usage guidance with exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mcp_vision_augment_visionA

多模态视觉工具：为无视觉能力的 LLM 提供看图、OCR 与文档解析能力。

返回统一 JSON envelope：{task_type, tool_used, code, error, result, confidence, metadata}。 视觉理解按配置的 OpenAI 兼容通道依次降级，最后兜底本地 Ollama VL 模型； OCR 与文档解析在本地完成（RapidOCR / markitdown）。

ParametersJSON Schema
NameRequiredDescriptionDefault
taskNoreasoning 时的提问内容（如“请描述这张图片”），其他任务忽略
sourceYes输入：本地文件路径 / file:// / http(s)://（需 VISION_AUGMENT_ALLOW_URLS=true）/ data:URL
languageNoOCR 语言：ch/en/japan/korea/latin，仅 ocr 任务生效ch
task_typeYes任务类型：reasoning=视觉理解（看图问答）；ocr=图像文字识别；document=文档解析（docx/pdf/pptx/xlsx/html/md）

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral transparency burden. It discloses the unified return envelope, the fallback chain through OpenAI-compatible channels to local Ollama VL, and that OCR/document parsing run locally. This adds meaningful context beyond the schema, though it does not mention rate limits, timeouts, or error code specifics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two succinct sentences: the first states the tool's role, the second explains the envelope and fallback behavior. It front-loads the main purpose, uses the space efficiently, and avoids redundancy with the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (multiple task types, multiple backends), and the description covers the envelope, backend fallback, and local processing. The schema fully documents all parameters. However, there is no per-task description of what 'result' contains, and the VISION_AUGMENT_ALLOW_URLS requirement appears only in the schema, not the description. Given the complexity, these are minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%—every parameter (task, source, language, task_type) has a clear description in the input schema. The description adds no extra parameter-level semantics, so the baseline of 3 is appropriate. The description's mention of the return envelope is about output structure, not parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb+resource: 'provides image viewing, OCR, and document parsing capabilities' for LLMs without vision. It clearly distinguishes itself from sibling tools (health, clear_cache) by enumerating the three task types (reasoning, ocr, document) and the unified JSON envelope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly implies when to use the tool (vision understanding, OCR, document parsing) via the task_type enum, and the fallback behavior indicates the intended scenarios. However, it does not explicitly state when not to use it or contrast with sibling tools, so it stops short of a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4/5.0
Disambiguation5/5

Each tool has a clearly distinct purpose: the vision tool performs vision/OCR/document tasks, clear_cache manages caching, and health checks the environment. There is no overlap or ambiguity between them.

Naming Consistency4/5

All tools share the consistent mcp_vision_augment_ prefix, but the suffixes are not uniform: 'clear_cache' follows verb_noun, while 'vision' and 'health' are nouns. Still, the pattern is predictable and readable.

Tool Count4/5

Three tools is at the lower end of the typical range but is reasonable for this server's scope: one main functional tool plus two support tools. It does not feel unnecessary or sparse.

Completeness5/5

The tool surface covers the core vision augmentation functionality (vision, OCR, document parsing) and the necessary operational tools (health check, cache clearing). No obvious missing operations that would impede an agent.

Maintenance

ActivityMaintained
ResponsivenessSyncing

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • picsense

    honlnk
    A
    license
    Not graded
    quality
    B
    maintenance
    Local MCP server that provides multi-modal vision capabilities to single-modal base models via API, supporting multi-turn iterative image recognition and document image parsing.
    125
    Apache 2.0

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/CaoMeiYouRen/vision-augment'

If you have feedback or need assistance with the MCP directory API, please join our Discord server