opencode-eyes-nvidia 👁️

MCP server that provides image description capability using NVIDIA NIM hosted API (https://integrate.api.nvidia.com/v1) with the MiniMax-M3 multimodal model.

Provides "eyes" for models that lack multimodal capabilities. Feed in an image and get a detailed textual description of it.

Features

Related MCP server: vision-mcp

Multimodal Models

The default model is minimaxai/minimax-m3 (text/image/video input → text output, 1M context, supports reasoning). It can be switched via NVIDIA_MODEL or the model parameter of describe_image:

Environment Variables

* At least one key is required: any of NVIDIA_API_KEY, NVIDIA_API_KEYS, or NVIDIA_API_KEY_1..N will do; they can be configured together (deduplicated and merged).

Key Rotation (Automatic Multi-Key Switching)

Supports configuring multiple API keys: requests use them in order, and automatically switch to the next key on error. The original key is not removed — it is simply moved to the end of the queue and reused once its quota refreshes. Retries are fully automatic, no manual intervention needed.

How it works:

1. On startup, all keys are merged into a single queue (deduplicated, in the order NVIDIA_API_KEY → NVIDIA_API_KEYS → NVIDIA_API_KEY_1..N).

2. Requests use the key at the front of the queue by default.

3. On a retryable error (HTTP 401 / 403 / 404 / 408 / 429 / 5xx, or a connection timeout): the front key is moved to the back of the queue, and after waiting NVIDIA_ROTATION_BACKOFF seconds, the next key is used to retry.

4. If all keys have been rotated through (a total of NVIDIA_ROTATION_MAX_RETRIES attempts) without success, only then is the last error thrown.

5. Queue state is preserved across calls — rate-limited keys stay at the back, and by the time they come around again, their quota has usually refreshed.

Example:

# 方式一：逗号分隔多个 Key
set NVIDIA_API_KEYS=nvapi-key-1,nvapi-key-2,nvapi-key-3

# 方式二：编号 Key
set NVIDIA_API_KEY_1=nvapi-key-1
set NVIDIA_API_KEY_2=nvapi-key-2

# 方式三：单个 Key（向后兼容）
set NVIDIA_API_KEY=nvapi-key-1

Pass the environment variables to the MCP service in the OpenCode configuration:

{
  "mcp": {
    "opencode-eyes-nvidia": {
      "type": "local",
      "command": ["python", "-m", "opencode_eyes_nvidia"],
      "enabled": true,
      "timeout": 120000,
      "environment": {
        "NVIDIA_API_KEYS": "{env:NVIDIA_API_KEYS}"
      }
    }
  }
}

Installation

pip install -r requirements.txt

Or install as a package:

pip install .

Running

# 设置环境变量（Windows，至少一种）
set NVIDIA_API_KEY=nvapi-你的key
# 或 set NVIDIA_API_KEYS=nvapi-key-1,nvapi-key-2

# 启动服务
python -m opencode_eyes_nvidia

Configuring in OpenCode

{
  "mcp": {
    "opencode-eyes-nvidia": {
      "type": "local",
      "command": ["python", "-m", "opencode_eyes_nvidia"],
      "enabled": true,
      "timeout": 120000,
      "environment": {
        "NVIDIA_API_KEYS": "{env:NVIDIA_API_KEYS}",
        "NVIDIA_API_KEY": "{env:NVIDIA_API_KEY}"
      }
    }
  }
}

Manual Testing

Verify via stdio without starting opencode:

$env:NVIDIA_API_KEY = "nvapi-xxx"
python -m opencode_eyes_nvidia

Then send an MCP JSON-RPC message from another terminal, for example:

{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"0.0.0"}}}
{"jsonrpc":"2.0","id":2,"method":"tools/list"}
{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"describe_image","arguments":{"image_path":"C:/path/to/photo.jpg"}}}

Differences from opencode-eyes

• API changed from StepFun to NVIDIA NIM (https://integrate.api.nvidia.com/v1)

• Default model changed from step-3.7-flash to MiniMax-M3 (minimaxai/minimax-m3)

• Added the list_vision_models tool and multi-model switching capability

• Added support for MiniMax-M3's thinking_mode reasoning control

• Added multi-API-key rotation: automatically switches to the next key on error, with the original key moved to the back of the queue to wait for quota refresh

License

MIT