# 👁️ OpenSight MCP

[![npm version](https://img.shields.io/npm/v/opensight-mcp.svg)](https://www.npmjs.com/package/opensight-mcp)
[![Node.js](https://img.shields.io/badge/node-%3E%3D18-green.svg)](https://nodejs.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![MCP](https://img.shields.io/badge/MCP-1.0-green)](https://modelcontextprotocol.io)

**Multi-backend AI vision for MCP agents.** Analyze images, screenshots, and documents using local Ollama models (private, uncensored) or cloud APIs (OpenAI, Google Gemini, OpenRouter). Works with any MCP-compatible coding agent.

## Quick Start

```bash
npx opensight-mcp@latest
```

| | Best for | Install |
|---|---|---|
| **[Local & Private](#-backends)** | Privacy-first, uncensored, no API keys | `ollama pull minicpm-v:latest` |
| **[Free Tier APIs](#-backends)** | Zero-cost cloud vision via Google/OpenRouter | Set `GOOGLE_API_KEY` or `OPENROUTER_API_KEY` |
| **[Paid APIs](#-backends)** | OpenAI GPT-4o, Claude Vision, any OpenAI-compatible | Set `OPENAI_API_KEY` |

## 🎯 Why OpenSight?

| **Local & Private** | **Free Tiers** | **Multi-Vendor** | **Agent-Native** |
|:---:|:---:|:---:|:---:|
| Ollama runs on your hardware. No data leaves your network. Uncensored models. | Google Gemini Flash and OpenRouter offer free vision tiers. Zero cost to start. | One tool, any backend. Swap providers with an env var — no code changes. | Purpose-built for MCP agents. Clipboard, file paths, URLs, base64 — all supported. |

## 📦 Installation

### Standard config (all MCP clients)

```json
{
  "mcpServers": {
    "opensight": {
      "command": "npx",
      "args": ["opensight-mcp@latest"],
      "env": {
        "OLLAMA_HOST": "127.0.0.1:11434",
        "VISION_MODEL": "minicpm-v:latest"
      }
    }
  }
}
```

<details>
<summary><b>OpenCode</b></summary>

Add to `~/.config/opencode/opencode.jsonc`:

```json
{
  "mcp": {
    "opensight": {
      "type": "local",
      "command": ["npx", "opensight-mcp@latest"],
      "enabled": true,
      "env": {
        "OLLAMA_HOST": "192.168.46.34",
        "VISION_MODEL": "minicpm-v:latest"
      }
    }
  }
}
```
</details>

<details>
<summary><b>Claude Code</b></summary>

```bash
claude mcp add opensight npx opensight-mcp@latest
```
</details>

<details>
<summary><b>Claude Desktop</b></summary>

Add to `claude_desktop_config.json`:

```json
{
  "mcpServers": {
    "opensight": {
      "command": "npx",
      "args": ["opensight-mcp@latest"],
      "env": {
        "OLLAMA_HOST": "127.0.0.1:11434",
        "VISION_MODEL": "minicpm-v:latest"
      }
    }
  }
}
```
</details>

<details>
<summary><b>Cursor</b></summary>

Add to `~/.cursor/mcp.json`:

```json
{
  "mcpServers": {
    "opensight": {
      "command": "npx",
      "args": ["opensight-mcp@latest"]
    }
  }
}
```
</details>

<details>
<summary><b>VS Code</b></summary>

```bash
code --add-mcp '{"name":"opensight","command":"npx","args":["opensight-mcp@latest"]}'
```
</details>

<details>
<summary><b>Windsurf / Cline / Other</b></summary>

Use the standard config above. Same pattern for all MCP-compatible clients.
</details>

<details>
<summary><b>Hermes</b></summary>

Add to `~/.hermes/config.yaml`:

```yaml
mcp_servers:
  opensight:
    command: "npx"
    args: ["-y", "opensight-mcp@latest"]
    env:
      OLLAMA_HOST: "127.0.0.1:11434"
      VISION_MODEL: "minicpm-v:latest"
```

Then reload:

```
/reload-mcp
```

Verify it's loaded:

```
Tell me which MCP-backed tools are available right now.
```
</details>

### Manual install (for development)

```bash
git clone https://github.com/Mr-JoE1/opensight-mcp.git
cd opensight-mcp
npm install
npm test
```

## 🛠️ Tools

<details open>
<summary><b>Vision Analysis</b></summary>

- **`vision.analyze_image`** — General-purpose image analysis. Accepts data URIs (clipboard paste), base64, URLs, or file paths. Configurable system prompt, model, temperature.
- **`vision.describe`** — UI/QA-focused screenshot analysis. Defaults to a QA system prompt that identifies errors, warnings, and layout issues.
- **`vision.clipboard`** — Read and analyze images directly from the OS clipboard. Uses `clipboardy` for cross-platform support (macOS/Windows built-in, Linux needs xclip/wl-clipboard).
- **`vision.find_images`** — Scan common directories (~/Downloads, ~/Pictures, /tmp) for recently modified images. Zero dependencies — pure Node.js fs.
</details>

<details>
<summary><b>OCR & Text</b></summary>

- **`vision.ocr`** — Extract text from images using VLM or Tesseract OCR. Supports structured JSON output with text block types.
- **`vision.find_text`** — Locate specific text in an image with bounding box coordinates. Supports fuzzy matching.
</details>

<details>
<summary><b>Operations</b></summary>

- **`vision.warmup`** — Pre-load the vision model into GPU VRAM to eliminate cold-start latency.
- **`vision.health`** — Check backend connection status and list available models.
</details>

## 🔌 Backends

Configure via environment variables. The default is **Ollama** (local, no API keys needed).

| Provider | Env Var | Free Tier | Best For |
|----------|---------|-----------|----------|
| **Ollama** | `OLLAMA_HOST` | ✅ (your hardware) | Privacy, uncensored, offline |
| **Google Gemini** | `GOOGLE_API_KEY` | ✅ (Flash 2.0) | Free tier, high accuracy |
| **OpenRouter** | `OPENROUTER_API_KEY` | ✅ (qwen-vl free) | Multi-model, free tier |
| **OpenAI** | `OPENAI_API_KEY` | ❌ | GPT-4o, best quality |

**Set the active provider:**

```bash
# Use Google Gemini (free tier)
export VISION_PROVIDER=google
export GOOGLE_API_KEY=your_key_here

# Use OpenAI (paid)
export VISION_PROVIDER=openai
export OPENAI_API_KEY=sk-...

# Use OpenRouter (free qwen-vl)
export VISION_PROVIDER=openrouter
export OPENROUTER_API_KEY=your_key_here

# Default: Ollama (local)
export OLLAMA_HOST=192.168.46.34:11434
```

## ⚙️ Configuration

All settings via environment variables:

| Variable | Default | Description |
|----------|---------|-------------|
| `VISION_ENDPOINT` | `local` | Endpoint mode: `local` (OLLAMA_HOST/PORT) or `global` (OLLAMA_GLOBAL_HOST/PORT) |
| `OLLAMA_HOST` | `192.168.46.34` | Ollama server host (local endpoint) |
| `OLLAMA_PORT` | `11434` | Ollama API port (local endpoint) |
| `OLLAMA_GLOBAL_HOST` | `192.168.46.34` | Ollama server host (global/WAN endpoint) |
| `OLLAMA_GLOBAL_PORT` | `11434` | Ollama API port (global/WAN endpoint) |
| `VISION_MODEL` | `minicpm-v:latest` | Default vision model |
| `OCR_MODEL` | `minicpm-v:latest` | Default OCR model |
| `VISION_PROVIDER` | `ollama` | Backend: ollama, openai, google, openrouter |
| `OPENAI_API_KEY` | — | OpenAI API key |
| `GOOGLE_API_KEY` | — | Google Gemini API key |
| `OPENROUTER_API_KEY` | — | OpenRouter API key |
| `MAX_TOKENS` | `2048` | Max response tokens |
| `VISION_WARMUP_ON_START` | `1` | Auto-warmup model on server start |
| `VISION_KEEP_ALIVE` | `10m` | Model keep-alive duration |
| `VISION_TIMEOUT_MS` | `120000` | Request timeout (ms) |
| `VISION_MAX_RETRIES` | `3` | Retry attempts on failure |

### Local vs Global Endpoints

Toggle between LAN and WAN endpoints with `VISION_ENDPOINT`:

**Local (default):**
```bash
export VISION_ENDPOINT=local
export OLLAMA_HOST=192.168.46.34
export OLLAMA_PORT=11434
```

**Global (DDNS/WAN):**
```bash
export VISION_ENDPOINT=global
export OLLAMA_GLOBAL_HOST=ih-tatweer.duckdns.org
export OLLAMA_GLOBAL_PORT=13313
```

OpenCode example with both endpoints configured:
```json
{
  "mcp": {
    "vision": {
      "type": "local",
      "command": ["node", "/path/to/vision-mcp.mjs"],
      "env": {
        "VISION_ENDPOINT": "local",
        "OLLAMA_HOST": "192.168.46.34",
        "OLLAMA_PORT": "11434",
        "OLLAMA_GLOBAL_HOST": "ih-tatweer.duckdns.org",
        "OLLAMA_GLOBAL_PORT": "13313",
        "VISION_MODEL": "minicpm-v:latest"
      }
    }
  }
}
```

## 🏃 CLI Usage

```bash
# Health check
npx opensight-mcp health
# or
vlm health

# Analyze an image
vlm describe --image screenshot.png --prompt "What errors are visible?"

# Extract text (OCR)
vlm ocr --image document.png --engine vlm

# Find text with coordinates
vlm find --image app.png --query "Submit button"
```

## 🤖 Recommended Models

| Model | Size | Best For |
|-------|------|----------|
| `minicpm-v:latest` | ~5.5 GB | **Default.** General analysis, OCR, UI. Fast and accurate. |
| `llava:7b` | ~4 GB | Lightweight fallback for limited VRAM |
| `gemini-2.0-flash` | Cloud | Free tier, Google quality |
| `qwen/qwen2.5-vl-32b-instruct:free` | Cloud | Free tier via OpenRouter |
| `gpt-4o` | Cloud | Best quality (paid) |

## 🔧 Development

```bash
git clone https://github.com/Mr-JoE1/opensight-mcp.git
cd opensight-mcp
npm install
npm test                 # 21 unit tests via node:test
npm run test:watch       # Watch mode
```

```
opensight-mcp/
├── vision-mcp.mjs          # Main MCP server (8 tools)
├── vlm.mjs                 # CLI tool
├── src/
│   ├── helpers.mjs         # Pure utility functions (tested)
│   └── providers.mjs       # Multi-backend abstraction
├── tests/
│   └── helpers.test.mjs    # 21 unit tests
└── opensight-wrapper.sh    # MCP wrapper with env defaults
```

## 📄 License

MIT — see [LICENSE](LICENSE).

---

**Made for coding agents. Private by default. Cloud when you need it.**