# llm-vision-mcp

A TypeScript MCP server that gives text-only LLMs image understanding through StepFun vision models.

This is useful when the primary model, such as GLM5.2 or DeepSeek V4, does not support image input. The model can call these MCP tools, receive text or structured visual analysis, then continue reasoning with the result.

## Tools

- `analyze_image`: general image understanding
- `extract_text_from_image`: OCR for screenshots, logs, documents, code, and UI text
- `diagnose_error_screenshot`: error screenshot and stack trace diagnosis
- `understand_technical_diagram`: architecture, flowchart, UML, ER, sequence, and network diagrams
- `analyze_data_visualization`: charts, tables, dashboards, and metrics screenshots
- `ui_to_artifact`: UI screenshot to implementation notes or design specs
- `ui_diff_check`: expected vs actual UI screenshot comparison

## Setup

```bash
npm install
cp .env.example .env
```

Set `STEPFUN_API_KEY` in the MCP client environment. Injecting env vars through the MCP client config is usually the most explicit and reliable setup.

Required:

```bash
STEPFUN_API_KEY=your_stepfun_api_key
```

Optional:

```bash
# standard | step_plan
STEPFUN_API_MODE=standard
STEPFUN_BASE_URL=https://api.stepfun.com/v1
STEPFUN_VISION_MODEL=step-1o-turbo-vision
STEPFUN_DEFAULT_DETAIL=high
STEPFUN_TIMEOUT_MS=120000
```

## Step Plan

Step Plan uses the same API key style but a different Base URL:

```bash
STEPFUN_API_MODE=step_plan
```

When `STEPFUN_API_MODE=step_plan` is set, defaults change to:

```bash
STEPFUN_BASE_URL=https://api.stepfun.com/step_plan/v1
STEPFUN_VISION_MODEL=step-3.7-flash
```

You can still override either value explicitly:

```bash
STEPFUN_API_MODE=step_plan
STEPFUN_BASE_URL=https://api.stepfun.com/step_plan/v1
STEPFUN_VISION_MODEL=step-3.7-flash
```

For backward compatibility, `STEPFUN_USE_STEP_PLAN=true` also enables Step Plan mode when `STEPFUN_API_MODE` is not set.

## Run

```bash
npm run build
npm run start
```

## Run With npx

After this package is published to npm:

```bash
npx -y llm-vision-mcp
```

The published package runs on Node.js and does not require Bun on the user's machine.

## MCP Client Config

Example:

```json
{
  "mcpServers": {
    "llm-vision-mcp": {
      "command": "node",
      "args": ["/Users/shaoyun/workdir/llm-vision-mcp/dist/index.js"],
      "env": {
        "STEPFUN_API_KEY": "your_stepfun_api_key",
        "STEPFUN_API_MODE": "step_plan",
        "STEPFUN_DEFAULT_DETAIL": "high"
      }
    }
  }
}
```

npm package example after publishing:

```json
{
  "mcpServers": {
    "llm-vision-mcp": {
      "command": "npx",
      "args": ["-y", "llm-vision-mcp"],
      "env": {
        "STEPFUN_API_KEY": "your_stepfun_api_key",
        "STEPFUN_API_MODE": "step_plan",
        "STEPFUN_DEFAULT_DETAIL": "high"
      }
    }
  }
}
```

## Image Inputs

Every single-image tool accepts:

```json
{
  "image": "/absolute/path/to/screenshot.png",
  "question": "What does this error mean?",
  "detail": "high"
}
```

The `image` field supports:

- local file path
- `file://` path
- `http://` or `https://` URL
- `data:image/...;base64,...` Data URL

`ui_diff_check` accepts two images:

```json
{
  "expected_image": "/absolute/path/to/expected.png",
  "actual_image": "/absolute/path/to/actual.png",
  "question": "Focus on layout and missing buttons.",
  "detail": "high"
}
```

## Notes

- Use `detail: "high"` for OCR, UI, diagrams, charts, and screenshots.
- Use `detail: "low"` for faster, cheaper coarse image understanding.
- StepFun supports JPG/JPEG, PNG, WebP, and static GIF image inputs.