# picident-mcp

**English** | [中文](README-zh.md)

> **picident is a pure MCP server** that gives text-only LLM agents vision: 4 MCP tools (`vision` / `ocr` / `list_models` / `providers`) describe screenshots, UI, charts, and photos through OpenAI-compatible / Anthropic / Gemini / Ollama providers with automatic fallback. No desktop app, no credentials in the repo.

## Why

Your agent (Reasonix, Claude, Cursor, …) cannot see images. picident-mcp is the eyes: it receives an image path/URL/data-URI via an MCP tool call, forwards it to a vision-capable model, and returns a text description the agent can reason about.

## Features

- **4 MCP tools**: `vision`, `ocr`, `list_models`, `providers` — with automatic provider fallback (main fails → next enabled provider)
- **Any provider**: OpenAI-compatible (OpenAI / DeepSeek / OpenRouter / Qwen / GLM / Kimi / Z.AI / MiniMax / Moonshot / self-hosted gateways), Anthropic, Gemini, Ollama (local)
- **Transports**: stdio (primary — the client manages the process lifecycle) + Streamable HTTP (`http://127.0.0.1:8001/mcp`)
- **Security first**: credentials live only in process environment variables (injected by your MCP client), never in files, never committed; HTTP mode binds loopback with Host/Origin validation and optional Bearer-token auth
- **Robust**: images auto-downscaled to a max 2048px long edge; config hot-reload (no restart); structured call logs

## Install from the Reasonix registry

picident-mcp is published on the [Reasonix skills registry](https://reasonix.io/skills/).
Your agent installs it directly:

> Install this Reasonix skill package from `https://github.com/HaoyueQin/picture-identification-MCP/blob/main-v2/skills/picident/SKILL.md`. Use `install_source` with kind="skill".

The [SKILL.md](skills/picident/SKILL.md) contains agent-executable steps: download the prebuilt
binary for your platform (`bin/picident-launcher.sh` / `bin/picident-launcher.ps1`
auto-download from GitHub Releases) or build from source, then wire it into your
MCP client.

## Quick Start

### 1. Build

```bash
cargo build --release -p picident-server
# → target/release/picident-server.exe (Windows) / picident-server (Linux/macOS)
```

### 2. Write config.toml

Copy [`config.example.toml`](config.example.toml) to your OS config directory:

- Windows: `%APPDATA%/picident/config.toml`
- Linux: `~/.config/picident/config.toml`
- macOS: `~/Library/Application Support/picident/config.toml`

The config defines **which providers exist** (protocol, base URL, model, context window). Credentials are configured separately — see below.

### 3. Connect your agent

Add to your agent's MCP configuration (`mcpServers`). **stdio** mode (recommended, local):

```json
{
  "mcpServers": {
    "picident": {
      "command": "C:\\path\\to\\picident-server.exe",
      "args": ["--stdio"],
      "env": {
        "PICIDENT__providers__openai__token": "sk-xxxx"
      }
    }
  }
}
```

> The `env` block is how the client injects credentials — it becomes the server process environment. This is the standard MCP mechanism (same as Claude Desktop, Cursor, Claude Code). The keys never touch your filesystem or git history.

**Streamable HTTP** mode (remote / LAN):

```json
{
  "mcpServers": {
    "picident": {
      "url": "http://127.0.0.1:8001/mcp"
    }
  }
}
```

```bash
# start the server with credentials in ITS environment:
PICIDENT__providers__openai__token=sk-xxxx ./picident-server --http --http-port 8001
```

## Configuring providers in detail

### Choosing the protocol: `kind`

| `kind` | Protocol | Typical base_url |
|--------|----------|------------------|
| `open_ai_compat` | OpenAI Chat Completions (works with DeepSeek, OpenRouter, Qwen, GLM, Kimi, Z.AI, MiniMax, most gateways) | `https://api.openai.com/v1`, `https://api.deepseek.com/v1`, `https://openrouter.ai/api/v1`, … |
| `anthropic` | Anthropic Messages API | `https://api.anthropic.com` |
| `gemini` | Google Generative Language API | `https://generativelanguage.googleapis.com` |
| `ollama` | Local Ollama (no API key) | `http://localhost:11434` |

```toml
[[providers]]
id = "deepseek"
kind = "open_ai_compat"          # ← protocol
label = "DeepSeek"
model = "deepseek-vl2"
max_output_tokens = 4096         # ← max output tokens
enabled = true
[providers.extra]
base_url = "https://api.deepseek.com/v1"   # ← endpoint URL
```

### Configuring the API key: environment variables

Every provider reads its credential from the process environment, keyed by provider `id`:

```text
PICIDENT__providers__<id>__token       # general token / API key
PICIDENT__providers__<id>__api_key     # alternative field (either is enough)
```

Rules:

- `id` in the variable name uses `_` in place of `-` (env vars cannot carry hyphens portably). A provider with `id = "z-ai"` uses `PICIDENT__providers__z_ai__token`.
- Ids that already contain `_` (e.g. `my_provider`) resolve by **exact match** against the configured ids, so their variables are never misread as the hyphenated spelling.
- Providers that don't need a key (e.g. local `ollama`) require no variables.
- Where to set them: the `env` block of your client's `mcpServers` config (stdio), or the shell/systemd environment of the server process (HTTP).
- **Never** put keys in `config.toml`, files, or git.

### Context window & token limits

| Field | Meaning | Default |
|-------|---------|---------|
| `max_output_tokens` | Max tokens in the response. | 4096 |
| `default_prompt` | Optional provider-level default prompt (used when the tool call has no `prompt`). | — |
| `model` | Default model; empty = built-in per-protocol default (e.g. `gpt-4o`, `claude-sonnet-4-6`, `gemini-2.5-flash`, `llava`). | `""` |

### Model-level primary/fallback

Instead of provider-level fallback (first enabled → next), you can pin model sequences:

```toml
[mcp]
primary = "openai/gpt-4o"
fallback = ["anthropic/claude-sonnet-4-6", "ollama/llava"]
```

Empty config falls back to "try every enabled provider in config order".

### Image limits

- Images are auto-downscaled to a **max 2048px long edge** (re-encoded as JPEG q85), so oversized screenshots never trip provider dimension limits.
- Input accepts local file paths, `http(s)` URLs, and `data:` URIs.
- URL downloads are **bounded**: max 32 MB response body, 60 s timeout; images whose declared dimensions exceed 50 megapixels are rejected before decoding (decompression-bomb guard).
- HTTP mode allows request bodies up to 32 MB.

## MCP tools

### `vision`

Describe screenshots, UI previews, charts, diagrams, photos. Pass one or more images (path / URL / data-URI) plus an optional `prompt` describing what you need.

**When to use:** any task where you (the agent) need to *see* something — frontend verification, analyzing an image file the user mentions, reading a chart.

### `ocr`

Extract ALL text from one image. Optional `language` hint (`zh` / `en` / `auto`). Returns only the extracted text.

### `list_models`

List models available from a provider (or all). Uses the configured credential server-side.

### `providers`

List configured providers and their readiness status (ready = enabled and loaded; skipped = configured but failed to load, e.g. missing credentials).

## Security posture

- **Credentials**: only in process env, injected by the MCP client. No secrets file, no repo leakage, redacted in all logs/Debug output.
- **HTTP mode**: binds `127.0.0.1` only; Host-header validation (DNS-rebinding defense); Origin validation for browser requests; optional `Authorization: Bearer <token>` via `[server].http_auth_token` — set it when exposing beyond localhost.
- **Call logs**: whitelisted fields only (image count/size, prompt preview ≤200 chars) — never base64, never credentials.
- **Environment-variable config**: `PICIDENT_HOME` / `PICIDENT_CONFIG_DIR` override the config dir; `PICIDENT__server__http_port` / `PICIDENT__server__http_auth_token` override the server section.

## Architecture

```
crates/
├── core/       # config, provider adapters, image normalization, logging
└── server/     # headless MCP server (rmcp: stdio + Streamable HTTP)

ImageInput → normalize() → base64 data-URI → Provider.describe()
                                            ↓
                              mpsc::Receiver<VisionEvent>
                                            ↓
                          Delta | Thinking | Usage | Done
```

## Development

```bash
cargo build -p picident-server
cargo test --workspace
cargo clippy --workspace -- -D warnings
```

## Acknowledgments

- [rmcp](https://github.com/nicholasgasior/rmcp) — Rust MCP SDK (stdio + Streamable HTTP)
- [esengine/DeepSeek-Reasonix](https://github.com/esengine/DeepSeek-Reasonix) — MCP model-level primary/fallback configuration pattern
- [astrbot_plugin_irmia_vision](https://github.com/) — 2048px image downscale approach

## License

[MIT](LICENSE) © HaoyueQin