gemini-vision-mcp

MCP server for describing video, image, audio, and text using the Gemini 3.1 Flash Live API.

Uses a persistent WebSocket session (Google GenAI Live API) to describe media content through FastMCP.

Requirements

• Python 3.11+

• Google API key with Gemini access

Related MCP server: Gemini MCP Server

Install

uv pip install gemini-vision-mcp

Or run directly with uvx:

uvx gemini-vision-mcp

Configuration

Copy .env.example to .env and fill in your API key:

cp .env.example .env

Usage

Run with uvx:

uvx --no-cache --from . gemini-vision-mcp

This project uses uvx for running and venv for development.

MCP Client Configuration

Add to your MCP client (e.g. opencode opencode.json):

{
  "mcp": {
    "gemini-vision": {
      "environment": {
        "GOOGLE_API_KEY": "your-api-key",
        "GEMINI_MODEL": "gemini-3.1-flash-live-preview",
        "RATE_LIMIT_WINDOW_SECONDS": "60",
        "RATE_LIMIT_REQUESTS": "10"
      },
      "enabled": true,
      "type": "local",
      "cwd": ".",
      "command": [
        "uvx",
        "--no-cache",
        "--from",
        ".",
        "gemini-vision-mcp"
      ]
    }
  }
}

Tools

Parameters

• type: Media category — text, image, audio, or video

• prompt: Instructions to the model (what to describe/extract)

• path: Local file path (file tools)

• resource: URL to fetch (resource tools)

• compact: If true, auto-compact history after this request (vision tools only)

Example: describe an image

vision_from_file(type="image", path="/path/to/photo.jpg", prompt="Describe the scene in detail")

Example: extract text from a screenshot

vision_from_file(type="image", path="/path/to/screenshot.png", prompt="Extract all visible text")

Example: batch describe images

vision_from_files(type="image", paths=["/a.jpg", "/b.jpg", "/c.jpg"], prompt="Describe each image")

Example: describe a URL resource

vision_from_resource(type="image", resource="https://example.com/photo.png", prompt="What is in this image?")

Session management

The server maintains a conversation history across tool calls and uses a two-tier TPM management strategy:

• 70% TPM — Auto-compaction: the model summarizes the conversation history, then reconnects with the summary as context. History shrinks, session continues.

• 75% TPM — Hard refresh: reconnects with whatever compacted history exists. Use new_vision to reset completely (loses all context).

You can also trigger compaction manually:

compact()

Or pass compact=true to any vision tool to compact after that request:

vision_from_file(type="image", path="photo.jpg", prompt="Describe this", compact=true)

Call new_vision to force a clean reset (no history carried over). Call gemini_usage to inspect current token consumption and rate limit status.

Architecture

• src/server.py — FastMCP server, tool registration, rate limiting

• src/gemini_live.py — Persistent Gemini Live WebSocket client with auto-reconnect, history tracking, and self-compaction

• src/rate_limiter.py — Sliding window rate limiter

• src/tool/ — Tool implementations (compact, file, files, files_mixed, gemini_usage, new_vision, resource, resources, resources_mixed)

• src/event/ — Background TPM monitoring