Makia Vision MCP — Media Intelligence for any MCP client

English | فارسی

One MCP server that gives your agent eyes and hands for media: deterministic video/image/audio engines (FFmpeg, sharp) plus pluggable AI providers (Gemini, Qwen3-VL, any OpenAI-compatible endpoint, whisper.cpp). Every answer is structured, timestamped, and evidence-backed — the server returns video_id + start_ms + end_ms windows you can re-verify with video_frame/video_clip.

video_analyze → { answer, events: [{start_ms, end_ms, description, confidence, evidence}] }
                → video_clip(evidence) → independent verification

Design rules (from SPEC.md)

1. Video is a first-class sibling of images, not a mode of image_analyze.

2. Time is canonical — integer milliseconds internally; HH:MM:SS.mmm at the edge. Never frame numbers.

3. Artifacts are immutable — every operation creates a new artifact with lineage. Never overwrites sources.

4. Every AI claim carries evidence — no evidence → marked unverified.

5. Deterministic first — if FFprobe/FFmpeg can answer, no AI call happens.

6. Privacy is structural — only *_analyze/video_transcribe may transmit media, only when cloud_policy allows.

7. No raw ffmpeg args — structured ops compile to FFmpeg invocations internally.

Related MCP server: pictor-mcp

Quickstart

Prereqs: Node ≥ 22.5 and ffmpeg/ffprobe on PATH (Windows: winget install ffmpeg; macOS: brew install ffmpeg; Debian/Ubuntu: sudo apt install ffmpeg; Arch: sudo pacman -S ffmpeg). Check with makia-vision-mcp --doctor.

// Standard config — works in Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, Cline, …
{
  "mcpServers": {
    "makia-vision": {
      "command": "npx",
      "args": ["-y", "makia-vision-mcp@latest"],
      "env": {
        // optional: enables AI analysis. All deterministic tools work without keys.
        "GEMINI_API_KEY": "AIza…"
      }
    }
  }
}

claude mcp add makia-vision -s user -e GEMINI_API_KEY=AIza… -- npx -y makia-vision-mcp@latest
# scopes: -s user (all projects) | -s project (committed .mcp.json) | -s local (this project only)

Build the bundle: npm pack && npx @anthropic-ai/mcpb pack packaging/mcpb/manifest.json — then open the resulting .mcpb file, or paste the standard config into claude_desktop_config.json.

Add to .vscode/mcp.json (workspace) or user settings — standard snippet above. For secrets prefer input variables: "inputs": [{ "name": "gemini", "type": "promptString", "password": true }].

~/.cursor/mcp.json (global) or .cursor/mcp.json (project) — standard snippet above; ${env:GEMINI_API_KEY} interpolation supported.

~/.gemini/settings.json → mcpServers block — standard snippet above.

{
  "mcpServers": {
    "makia-vision": {
      "command": "npx", "args": ["-y", "makia-vision-mcp@latest"],
      "env": {
        "MAKIA_CLOUD_POLICY": "LOCAL",
        // any OpenAI-compatible local server (LM Studio :1234, llama.cpp, vLLM, Ollama /v1):
        "OPENAI_BASE_URL": "http://127.0.0.1:1234/v1",
        "WHISPER_CPP_BIN": "/usr/local/bin/whisper-cli",
        "WHISPER_CPP_MODEL": "/models/ggml-base.bin"
      }
    }
  }
}

MAKIA_CLOUD_POLICY=LOCAL guarantees media never leaves the machine; the mock-friendly openai-compat path covers local multimodal servers.

Tools (v1 — see media_capabilities for live status)

Storage

$MAKIA_VISION_HOME (default ~/.makia-vision)
  artifacts/<vid_|img_|aud_|trn_|scm_…>/meta.json + data.<ext>
  tmp/            # scratch, swept at startup + TTL
  route-log.ndjson # routing/provider decisions (observability)

Configuration

// ~/.makia-vision/config.json (all optional)
{
  "models": {
    "image":         { "provider": "gemini", "model": "gemini-3.8-flash" },
    "video":         { "provider": "qwen", "model": "qwen3-vl-plus" },
    "transcription": { "provider": "openai", "model": "whisper-1" },
    "cloud_policy":  "HYBRID"
  },
  "security": {
    "max_duration_s": 14400, "max_width": 8192, "max_height": 8192,
    "max_pixels_per_frame": 8294400, "max_total_pixels": 497664000,
    "max_streams": 64, "max_bytes": 21474836480, "probe_timeout_ms": 30000, "op_timeout_s": 600
  },
  "keep_artifacts_h": 72
}

Env: MAKIA_VISION_HOME, MAKIA_CLOUD_POLICY, GEMINI_API_KEY/GOOGLE_API_KEY, OPENAI_API_KEY, OPENAI_BASE_URL, QWEN_API_KEY/DASHSCOPE_API_KEY, WHISPER_CPP_BIN, WHISPER_CPP_MODEL, MAKIA_VISION_FFMPEG_DIR (air-gapped), HTTPS_PROXY (fetcher/provider calls).

Universal install matrix

Security posture

• Decode budgets before any decode: duration/width/height/total-pixels/streams/bytes caps; probing under strict timeouts (malformed-container abuse).

• URL ingestion through a hardened fetcher: scheme allowlist, DNS pinning, private-network denial (CGNAT/link-local/loopback), redirect validation, size caps. Never ffmpeg -i URL.

• Immutability: ops never overwrite sources; exports refuse to overwrite existing files.

• Prompt injection: images/video may contain adversarial content; the server returns structured evidence (not free-text instructions) and marks unverified claims.

• Error taxonomy: structured {error:{code,message}} across the MCP boundary — never raw stack traces.

Development

npm install        # sharp prebuilts fetched per-platform
npm run build      # tsc → dist/
npm test           # unit + ffmpeg integration + MCP stdio e2e
npm run dev        # stdio via tsx
node dist/index.js --http --port 8629   # Streamable HTTP on 127.0.0.1
node dist/index.js --doctor             # capability health check

License: project code MIT — see LICENSE-NOTES.md for the FFmpeg/libvips/provider license chain. Architecture: SPEC.md · research: RESEARCH.md · plan: PLAN.md.