dsh-vision

A tool that provides visual capabilities for DeepSeek Harness (dsh) plugins and Claude Code (MCP): local OCR (macOS / Windows) + cloud VLM (multi-vendor) image understanding.

DeepSeek's model API currently does not support image input, so read_image is unavailable. This plugin provides two tools to work around this limitation:

Features

• 🔒 Local OCR: macOS is based on Vision.framework, Windows on the built-in OCR engine; images never leave your machine, keeping your privacy safe

• ☁️ Cloud VLM: Defaults to Alibaba Cloud Bailian qwen3-vl-flash (fast and cheap), OpenAI-compatible API, can be swapped for any provider

• 🖼️ Auto compression: Uses sips to compress large images to 2048px / JPEG 85% before VLM calls, saving money and bandwidth

• 🔑 Flexible key retrieval: Environment variable or ~/.dsh/.credentials.yaml

• 🧪 Zero-dependency unit tests: Core logic covered with Node's built-in node:test (13 test cases)

• 🌏 Multi-vendor VLM: Built-in Bailian / SiliconFlow / Zhipu / Volcano Ark, OpenAI-compatible so any vendor can be added

• 🔌 Dual form: Works as both a dsh plugin and an MCP server (usable directly by MCP clients such as Claude Code)

• 🪟 Windows support: Includes a PowerShell OCR backend (Windows.Media.Ocr); the VLM channel is cross-platform

Installation

Method A: Install from npm (recommended)

dsh plugin --profile web add @floatingsk/dsh-vision

Method B: Copy from source

# 把本仓库拷贝到你的 dsh profile 插件目录
cp -R dsh-vision ~/.dsh/profiles/node_modules/dsh-vision

Method C: Download precompiled binaries from GitHub Release (no compilation needed, recommended)

When maintainers push a v* tag, GitHub Actions automatically compiles on both macOS architectures and attaches the binaries to the Release:

1. Open the Releases page of this repository and select the latest version

2. Download according to your Mac's architecture:

   • Apple Silicon (M series): vision-ocr-arm64

   • Intel Mac: vision-ocr-x86_64

3. Place it in the plugin directory and add execute permission:

cp vision-ocr-arm64 ~/.dsh/profiles/node_modules/dsh-vision/bin/vision-ocr
chmod +x ~/.dsh/profiles/node_modules/dsh-vision/bin/vision-ocr

Compile the OCR binary (macOS requires Xcode Command Line Tools)

cd ~/.dsh/profiles/node_modules/dsh-vision
# 显式指定 clang 模块缓存目录（沙箱/受限环境下必需）
swiftc -Xcc -fmodules-cache-path="$PWD/.cache" -O bin/vision-ocr.swift -o bin/vision-ocr

Enable the plugin in a profile patch

Edit ~/.dsh/profiles/web/cordis.patch.yml (the file corresponding to your profile) and append:

- insert:
    - id: dsh-vision
      name: 'dsh-vision'

Configure the VLM API Key (required for describe_image)

Choose one of the following:

# 方式 A：环境变量
export DASHSCOPE_API_KEY=sk-xxx

# 方式 B：写入 dsh 凭据文件
echo 'DASHSCOPE_API_KEY: sk-xxx' >> ~/.dsh/.credentials.yaml

Get the key from your VLM provider's console (default is Alibaba Cloud Bailian: bailian.console.aliyun.com).

Restart dsh

The tools become available after restarting. Note: you need to start a new conversation; the tool list is injected when the session begins.

Usage

Save the image to disk during the conversation and tell the agent its path:

看下 /path/to/image.png 里有什么
读取 /path/to/截图.png 中的文字

The agent will automatically pick the appropriate tool (text reading goes through OCR, image understanding goes through VLM). To specify a VLM vendor, ask the agent to pass the provider parameter (e.g. bailian / siliconflow / zhipu / volcengine).

Configuration

Override default values through the config of the dsh-vision node in cordis.patch.yml.

Multi-vendor VLM

Four domestic vendors are built in. describe_image accepts a provider parameter to select one (leave blank to use defaultProvider):

- insert:
    - id: dsh-vision
      name: 'dsh-vision'
      config:
        defaultProvider: 'bailian'          # 默认供应商
        providers:
          bailian:                          # 阿里云百炼
            baseUrl: 'https://dashscope.aliyuncs.com/compatible-mode/v1'
            model: 'qwen3-vl-flash'         # 或 qwen3-vl-plus / qwen-vl-ocr
            apiKeyEnv: 'DASHSCOPE_API_KEY'
          siliconflow:                      # 硅基流动
            baseUrl: 'https://api.siliconflow.cn/v1'
            model: 'Qwen/Qwen2.5-VL-7B-Instruct'
            apiKeyEnv: 'SILICONFLOW_API_KEY'
          zhipu:                            # 智谱
            baseUrl: 'https://open.bigmodel.cn/api/paas/v4'
            model: 'glm-4v-flash'
            apiKeyEnv: 'ZHIPU_API_KEY'
          volcengine:                       # 火山方舟（豆包）
            baseUrl: 'https://ark.cn-beijing.volces.com/api/v3'
            model: 'doubao-seed-1.6-vision'
            apiKeyEnv: 'ARK_API_KEY'
        # 自定义 OCR 二进制路径（默认插件 bin/vision-ocr）
        ocrBin: ''
        # 上传前压缩最长边（像素）
        vlmMaxImageDim: 2048

To switch vendors: change defaultProvider, or specify the provider parameter when calling. To add a new vendor: add any key name under providers (any OpenAI-compatible endpoint works).

Recommended vision models (Alibaba Cloud Bailian)

Claude Code / MCP usage

This repository includes a zero-dependency MCP server (mcp/server.js) that lets Claude Code (and any MCP-compatible client) use these two tools—even if your Claude Code is connected to a model that does not support vision (such as DeepSeek).

Connect to Claude Code

# 全局接入（所有项目可用）
claude mcp add dsh-vision -- node /path/to/dsh-vision/mcp/server.js

# 或者只给当前项目（在项目根目录建 .mcp.json）：
# {
#   "mcpServers": {
#     "dsh-vision": {
#       "command": "node",
#       "args": ["/path/to/dsh-vision/mcp/server.js"],
#       "env": { "DASHSCOPE_API_KEY": "sk-xxx" }
#     }
#   }
# }

describe_image's API key read priority: environment variable > ~/.dsh/.credentials.yaml. When using .mcp.json, you can configure it directly in env.

Verification

claude mcp list        # 应看到 dsh-vision
claude mcp test dsh-vision   # 或直接问 Claude：看下 /path/to/xxx.png 里是什么

The MCP server is a pure Node implementation (stdio JSON-RPC), has no third-party dependencies, and only requires Node >= 18.

Development

# 运行单元测试
node --test test/

# 重新编译 OCR 二进制
swiftc -Xcc -fmodules-cache-path="$PWD/.cache" -O bin/vision-ocr.swift -o bin/vision-ocr

Platform support

• macOS OCR: Depends on Vision.framework. The repository does not include compiled artifacts (see .gitignore):

  • Apple Silicon: self-compile with npm run build:ocr, or download vision-ocr-arm64 from GitHub Release

  • Intel: self-compile with npm run build:ocr, or download vision-ocr-x86_64 from the Release

  • When pushing a v* tag to GitHub, Actions automatically compiles on both architectures and attaches the binaries to the Release

• Windows OCR: bin/vision-ocr.ps1 (Windows 10/11 built-in OCR engine; requires the Chinese OCR language pack). Just point the plugin's ocrBin to it:

  powershell -ExecutionPolicy Bypass -File bin/vision-ocr.ps1 <image> -Json

  Note: This script was developed on macOS and has not been tested on Windows; issues/PRs are welcome.

• VLM channel: Node >= 18 (built-in fetch), available on all platforms.

License

MIT