# Makia Vision MCP — Media Intelligence for any MCP client

**English** | [فارسی](README.fa.md)

One MCP server that gives your agent **eyes and hands for media**: deterministic video/image/audio engines (FFmpeg, sharp) plus pluggable AI providers (Gemini, Qwen3-VL, any OpenAI-compatible endpoint, whisper.cpp). Every answer is structured, timestamped, and **evidence-backed** — the server returns `video_id + start_ms + end_ms` windows you can re-verify with `video_frame`/`video_clip`.

```
video_analyze → { answer, events: [{start_ms, end_ms, description, confidence, evidence}] }
                → video_clip(evidence) → independent verification
```

## Design rules (from SPEC.md)

1. **Video is a first-class sibling of images**, not a mode of `image_analyze`.
2. **Time is canonical** — integer milliseconds internally; `HH:MM:SS.mmm` at the edge. Never frame numbers.
3. **Artifacts are immutable** — every operation creates a new artifact with lineage. Never overwrites sources.
4. **Every AI claim carries evidence** — no evidence → marked unverified.
5. **Deterministic first** — if FFprobe/FFmpeg can answer, no AI call happens.
6. **Privacy is structural** — only `*_analyze`/`video_transcribe` may transmit media, only when `cloud_policy` allows.
7. **No raw ffmpeg args** — structured ops compile to FFmpeg invocations internally.

## Quickstart

Prereqs: **Node ≥ 22.5** and **ffmpeg/ffprobe on PATH** (Windows: `winget install ffmpeg`; macOS: `brew install ffmpeg`; Debian/Ubuntu: `sudo apt install ffmpeg`; Arch: `sudo pacman -S ffmpeg`). Check with `makia-vision-mcp --doctor`.

```jsonc
// Standard config — works in Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, Cline, …
{
  "mcpServers": {
    "makia-vision": {
      "command": "npx",
      "args": ["-y", "makia-vision-mcp@latest"],
      "env": {
        // optional: enables AI analysis. All deterministic tools work without keys.
        "GEMINI_API_KEY": "AIza…"
      }
    }
  }
}
```

<details><summary><b>Claude Code</b></summary>

```bash
claude mcp add makia-vision -s user -e GEMINI_API_KEY=AIza… -- npx -y makia-vision-mcp@latest
# scopes: -s user (all projects) | -s project (committed .mcp.json) | -s local (this project only)
```
</details>

<details><summary><b>Claude Desktop (one-click)</b></summary>

Build the bundle: `npm pack && npx @anthropic-ai/mcpb pack packaging/mcpb/manifest.json` — then open the resulting `.mcpb` file, or paste the standard config into `claude_desktop_config.json`.
</details>

<details><summary><b>VS Code / GitHub Copilot</b></summary>

Add to `.vscode/mcp.json` (workspace) or user settings — standard snippet above. For secrets prefer input variables: `"inputs": [{ "name": "gemini", "type": "promptString", "password": true }]`.
</details>

<details><summary><b>Cursor</b></summary>

`~/.cursor/mcp.json` (global) or `.cursor/mcp.json` (project) — standard snippet above; `${env:GEMINI_API_KEY}` interpolation supported.
</details>

<details><summary><b>Gemini CLI</b></summary>

`~/.gemini/settings.json` → `mcpServers` block — standard snippet above.
</details>

<details><summary><b>Fully local (zero cloud)</b></summary>

```jsonc
{
  "mcpServers": {
    "makia-vision": {
      "command": "npx", "args": ["-y", "makia-vision-mcp@latest"],
      "env": {
        "MAKIA_CLOUD_POLICY": "LOCAL",
        // any OpenAI-compatible local server (LM Studio :1234, llama.cpp, vLLM, Ollama /v1):
        "OPENAI_BASE_URL": "http://127.0.0.1:1234/v1",
        "WHISPER_CPP_BIN": "/usr/local/bin/whisper-cli",
        "WHISPER_CPP_MODEL": "/models/ggml-base.bin"
      }
    }
  }
}
```
`MAKIA_CLOUD_POLICY=LOCAL` guarantees media never leaves the machine; the mock-friendly `openai-compat` path covers local multimodal servers.
</details>

## Tools (v1 — see `media_capabilities` for live status)

| Tool | Deterministic | Description |
|---|---|---|
| `video_open` / `image_open` | ✔ | Import from path or SSRF-hardened URL → `vid_…` / `img_…` |
| `video_inspect` | ✔ | Codec/streams/duration/fps/chapters/metadata |
| `video_frame` | ✔ | Frames at exact timestamps or uniform sampling → `img_…` with provenance |
| `video_scenes` | ✔ | FFmpeg scene detection → `scm_…` scene map (+representative frames) |
| `video_clip` | ✔ | Frame-accurate `[start_ms,end_ms)` extraction → new artifact |
| `video_transform` / `video_transcode` | ✔ | crop/resize/rotate/fps/speed; mp4/webm/mkv |
| `video_audio` | ✔ | Extract audio track → `aud_…` (wav/mp3/aac/flac) |
| `video_transcribe` | ✔/AI | Subtitle streams → whisper.cpp → cloud ASR → `trn_…` |
| `video_subtitles` | ✔ | extract / import / export (srt/vtt) |
| `video_compare` | ✔ | Stream properties + PSNR/SSIM |
| `video_export` / `image_export` | ✔ | Export payload (never overwrites) |
| `image_inspect` | ✔ | Dimensions + **dominant color palette** + exact pixel color |
| `image_edit` | ✔ | resize/crop/rotate/flip/convert/watermark/blur/sharpen/modulate/pad → new artifact |
| `image_analyze` | AI | Vision-provider analysis of any image artifact |
| `video_analyze` | AI | 14 modes (summary/timeline/locate_event/ocr/security/…), fast/balanced/deep, adaptive coarse→fine |
| `video_timeline` | AI | Structured event timeline (optionally focused) |
| `media_list` / `media_delete` / `media_capabilities` | ✔ | Store management + live capability report |

## Storage

```
$MAKIA_VISION_HOME (default ~/.makia-vision)
  artifacts/<vid_|img_|aud_|trn_|scm_…>/meta.json + data.<ext>
  tmp/            # scratch, swept at startup + TTL
  route-log.ndjson # routing/provider decisions (observability)
```

## Configuration

```jsonc
// ~/.makia-vision/config.json (all optional)
{
  "models": {
    "image":         { "provider": "gemini", "model": "gemini-3.8-flash" },
    "video":         { "provider": "qwen", "model": "qwen3-vl-plus" },
    "transcription": { "provider": "openai", "model": "whisper-1" },
    "cloud_policy":  "HYBRID"
  },
  "security": {
    "max_duration_s": 14400, "max_width": 8192, "max_height": 8192,
    "max_pixels_per_frame": 8294400, "max_total_pixels": 497664000,
    "max_streams": 64, "max_bytes": 21474836480, "probe_timeout_ms": 30000, "op_timeout_s": 600
  },
  "keep_artifacts_h": 72
}
```

Env: `MAKIA_VISION_HOME`, `MAKIA_CLOUD_POLICY`, `GEMINI_API_KEY`/`GOOGLE_API_KEY`, `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `QWEN_API_KEY`/`DASHSCOPE_API_KEY`, `WHISPER_CPP_BIN`, `WHISPER_CPP_MODEL`, `MAKIA_VISION_FFMPEG_DIR` (air-gapped), `HTTPS_PROXY` (fetcher/provider calls).

## Universal install matrix

| Platform | Status |
|---|---|
| Windows x64 / ARM64 | npm prebuilt via `@img/sharp-win32-*`; ffmpeg via winget/MSYS2 |
| macOS Intel / Apple Silicon | npm prebuilt via `@img/sharp-darwin-*`; ffmpeg via brew |
| Linux x64 / ARM64 / armv7 (glibc, musl) | npm prebuilt via `@img/sharp-linux-*`; ffmpeg via package manager |
| Offline / air-gapped | `npm pack` tarball + `MAKIA_VISION_FFMPEG_DIR` pointing at a pinned runtime; `cloud_policy: RESTRICTED` |

## Security posture

- **Decode budgets** before any decode: duration/width/height/total-pixels/streams/bytes caps; probing under strict timeouts (malformed-container abuse).
- **URL ingestion** through a hardened fetcher: scheme allowlist, DNS pinning, private-network denial (CGNAT/link-local/loopback), redirect validation, size caps. Never `ffmpeg -i URL`.
- **Immutability**: ops never overwrite sources; exports refuse to overwrite existing files.
- **Prompt injection**: images/video may contain adversarial content; the server returns structured evidence (not free-text instructions) and marks unverified claims.
- **Error taxonomy**: structured `{error:{code,message}}` across the MCP boundary — never raw stack traces.

## Development

```bash
npm install        # sharp prebuilts fetched per-platform
npm run build      # tsc → dist/
npm test           # unit + ffmpeg integration + MCP stdio e2e
npm run dev        # stdio via tsx
node dist/index.js --http --port 8629   # Streamable HTTP on 127.0.0.1
node dist/index.js --doctor             # capability health check
```

License: project code MIT — see [LICENSE-NOTES.md](./LICENSE-NOTES.md) for the FFmpeg/libvips/provider license chain. Architecture: [SPEC.md](./SPEC.md) · research: [RESEARCH.md](./RESEARCH.md) · plan: [PLAN.md](./PLAN.md).