Skip to main content
Glama
Sign In
deenesjakoruzh
ollama-vision-mcp

ollama-vision-mcp

by masterLazy
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

ollama-vision-mcp

English · 中文

Un servicio de puente mínimo que proporciona capacidades de visión local para VS Code Copilot utilizando modelos de solo texto (p. ej., DeepSeek).

Cuando Copilot utiliza un modelo de solo texto, no puede "ver" imágenes directamente. Este servidor MCP cubre esa carencia: colocas una captura de pantalla en una carpeta; Copilot lee la imagen a través de este puente → la envía a un modelo de visión local de Ollama → recibe una descripción de texto, que el modelo de solo texto puede entender.

VS Code Copilot Chat (Agent mode, text-only model)
        │  MCP stdio
        ▼
ollama-vision-mcp (this package)
        │  Read local image → base64 → POST /v1/chat/completions
        ▼
Ollama (local vision model, e.g., qwen2.5vl:7b)

Herramientas

Herramienta

Parámetros

Descripción

describe_image

path (obligatorio), mode?, question?

Lee una imagen y genera una descripción de texto. mode es opcional: general (predeterminado), ocr, ui, diagram; question puede proporcionar una consulta adicional.

list_images

directory? (predeterminado: directorio de inbox)

Lista los archivos de imagen disponibles para lectura.

extract_text

path (obligatorio)

Extrae texto de la imagen mediante OCR.

vision_status

—

Muestra la configuración actual, el estado de la conexión con Ollama y la lista de modelos disponibles.

Related MCP server: Hybrid Vision MCP Server

Alcance de esta herramienta

Este paquete es solo una capa de puente. Se comunica con Ollama puramente mediante HTTP (compatible con OpenAI /v1/chat/completions y /v1/models).

No se encarga de:

  • Instalar Ollama, iniciar ollama serve, descargar modelos o gestionar la configuración de los modelos;

  • Acceder al portapapeles, mecanismos internos del IDE o cualquier servicio de red que no sea tu Ollama local.

Eres totalmente responsable de instalar Ollama y descargar los modelos (consulta Requisitos previos).

Requisitos previos

Debes configurar lo siguiente por tu cuenta:

  • Python 3.10+

  • Ollama instalado y en ejecución: ollama serve

  • Al menos un modelo de visión descargado; recomendado:

    bash

    ollama pull qwen2.5vl:7b

    Otras opciones: qwen3-vl:8b, gemma3:12b, llama3.2-vision:11b, llava:7b. Elige un modelo adecuado para tu GPU; especifícalo más adelante mediante la variable de entorno VISION_MCP_MODEL.

🚀 Instalación rápida (recomendada)

Ejecuta un solo comando en la raíz del repositorio para completar la instalación y la configuración:

bash

cd ollama-vision-mcp
python setup_mcp.py

El script automáticamente:

  1. Crear un entorno virtual dedicado (.venv) e instalar este paquete (sin contaminar el entorno global).

  2. Detectar el servicio local de Ollama y listar los modelos de visión disponibles.

  3. Guiarte de forma interactiva para establecer base_url, model, inbox, max_tokens, la compresión de imagen y una clave de API opcional.

  4. Escribir la configuración en el .vscode/mcp.json del proyecto y/o en el archivo MCP global a nivel de usuario (%APPDATA%\Code\User\mcp.json), fusionándose con la configuración existente y sin sobrescribir tus otros servidores MCP.

Uso no interactivo (adecuado para CI/scripts):

bash

python setup_mcp.py --yes --project --model qwen2.5vl:7b
python setup_mcp.py --print         # Only prints the config JSON, does not write to file

Después de la instalación, recarga la ventana en VS Code (Ctrl+Shift+P → “Desarrollador: Recargar ventana”) para que la configuración surta efecto.

Si ya lo has ejecutado antes, también puedes reconfigurarlo simplemente con el comando ollama-vision-setup.

Verificar la instalación

En VS Code Copilot Chat (modo agente), introduce:

text

Run vision_status

Si el JSON devuelto contiene "ollama": { "ok": true, ... } y la lista de modelos, la conexión es correcta. Si ok es false, inicia primero Ollama (ollama serve) e inténtalo de nuevo.

Uso

  1. Coloca capturas de pantalla en el directorio de inbox del proyecto (por defecto .ai/inbox). El servidor creará este directorio automáticamente cuando sea necesario.

  2. Haz una pregunta en Copilot Chat, por ejemplo:

    “Mira las capturas de pantalla en el inbox y dime de qué trata este error.”

  3. El agente llamará automáticamente a list_images → describe_image y responderá basándose en la descripción de texto.

Haz que el agente sea más inteligente (opcional): Copia el archivo de instrucciones ya preparado .github/instructions/ollama-vision/vision-tools.instructions.md en la misma ruta de tu proyecto. Enseña al agente el flujo de trabajo completo de visión — orden de llamada (list_images → describe_image → extract_text), selección de modo, resolución de problemas y cómo guiarte para colocar capturas de pantalla en el inbox. Como instrucción de archivo de VS Code, se descubre bajo demanda siempre que la tarea implique imágenes, por lo que funciona directamente sin configuración.

Referencia de variables de entorno

Variable

Predeterminado

Descripción

VISION_MCP_BASE_URL

http://localhost:11434/v1

URL base compatible con OpenAI de Ollama

VISION_MCP_MODEL

qwen2.5vl:7b

Modelo de visión a utilizar

VISION_MCP_API_KEY

vacío (en realidad usa ollama)

Clave de API (ignorada por Ollama)

VISION_MCP_INBOX

.ai/inbox

Directorio predeterminado para list_images

VISION_MCP_MAX_TOKENS

2048

Máximo de tokens de salida para el modelo de visión

VISION_MCP_COMPRESS

1

Si la imagen > 50KB, redimensionar automáticamente a JPEG de 768px

VISION_MCP_THINK

0

Mantener el pensamiento habilitado para modelos de razonamiento (1), o deshabilitarlo mediante reasoning_effort=none (0, predeterminado)

Alias de compatibilidad: VISION_MODEL, VISION_BASE_URL, VISION_INBOX, VISION_API_KEY.

Instalación manual (alternativa)

Si necesitas control manual completo sobre cada paso, consulta el proceso a continuación.

1. Crear un entorno e instalar

bash

cd ollama-vision-mcp
python -m venv .venv
.venv\Scripts\activate          # Windows
# source .venv/bin/activate    # macOS / Linux
pip install -e .

Después de la instalación, tienes dos comandos:

  • ollama-vision-mcp — inicia el servidor MCP

  • ollama-vision-setup — acceso directo de configuración interactiva

2. Registrar con VS Code

Copia lo siguiente en el .vscode/mcp.json de tu proyecto (solo proyecto) o en el archivo a nivel de usuario %APPDATA%\Code\User\mcp.json (todos los proyectos). Asegúrate de cambiar command a la ruta absoluta del intérprete de Python de tu entorno virtual:

json

{
  "servers": {
    "ollama-vision": {
      "type": "stdio",
      "command": "D:/MyRepos/ollama-vision-mcp/.venv/Scripts/python.exe",
      "args": ["-m", "ollama_vision_mcp"],
      "env": {
        "VISION_MCP_BASE_URL": "http://localhost:11434/v1",
        "VISION_MCP_MODEL": "qwen2.5vl:7b",
        "VISION_MCP_INBOX": ".ai/inbox"
      }
    }
  }
}

⚠️ command debe ser la ruta absoluta al intérprete de Python dentro del entorno virtual. También puedes ejecutar ollama-vision-setup --project para generar este archivo automáticamente. El servidor utiliza os.getcwd() cuando se invoca, por lo que las rutas relativas (como path y el directorio de inbox) se resolverán en relación con la raíz del proyecto donde se inicia el servidor.

Prueba de humo

Ejecuta el siguiente script para comprobar la conectividad con Ollama, listar el contenido del inbox y realizar una llamada de visión real a la primera imagen:

bash

python examples/smoke_test.py

Licencia

MIT

Available Tools

4 tools
describe_imageA

Read a local image and return a text description from the local Ollama vision model.

Use when the user references a screenshot or image you cannot see. path is absolute, or relative to the project directory.

NameRequiredDescriptionDefault
modeNogeneral
pathYes
questionNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the local Ollama model and path semantics, but does not explain behavior around missing files, model limitations, or the purpose of mode/question parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose, followed by usage context and path semantics. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although an output schema exists, the description omits crucial parameter semantics for `mode` and `question`, which directly affect tool behavior. Without these details, an agent cannot fully understand how to use the tool for specialized cases like OCR or diagram analysis.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It explains only the `path` parameter ('absolute, or relative to the project directory'), leaving `mode` and `question` completely unexplained. This is a significant gap for a tool with three parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Read a local image and return a text description from the local Ollama vision model.' This specifies the action, resource, and method, and distinguishes it from siblings like extract_text by focusing on generating a description rather than extracting text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use when the user references a screenshot or image you cannot see.' This gives a clear condition for use, though it does not explicitly mention alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_textA

Extract all visible text (OCR) from a local image verbatim.

Use for code screenshots, terminal output, or error dialogs.

NameRequiredDescriptionDefault
pathYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. It states that OCR is used, that all visible text is extracted, and that output is verbatim, which are meaningful behavioral traits. It does not cover failure modes or image format limitations, but the core behavior is clearly disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no redundancy. The first sentence states the action and scope, and the second lists concrete use cases. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with an output schema and no complex side effects, the description covers the essential operational context: what it does, when to use it, and the key behavioral constraint (verbatim OCR). It is not exhaustive but adequately complete for its low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the only parameter is 'path' and the description's phrase 'from a local image' clearly establishes that path refers to a local image file. This compensates for the lack of schema documentation, though it does not specify path syntax or accepted formats.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Extract all visible text (OCR) from a local image verbatim.' It clearly distinguishes this tool from siblings like describe_image by focusing on text extraction rather than image description, and adds qualifiers ('all visible', 'verbatim') that make the scope precise.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives concrete use cases: 'Use for code screenshots, terminal output, or error dialogs.' This provides clear context for when to use the tool, though it does not explicitly mention alternatives or exclusions relative to sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_imagesA

List image files in a directory.

Defaults to the configured inbox folder where the user drops screenshots. Use this before describe_image to find what is available.

NameRequiredDescriptionDefault
directoryNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry behavioral transparency. It discloses the default behavior of falling back to the configured inbox folder, which is valuable. However, it does not specify whether subdirectories are included, result sorting, or other behavioral traits, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise, front-loaded sentences. Every word adds value, with no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple listing tool with an output schema present, the description covers purpose, default behavior, and usage guidance. It is sufficiently complete for an agent to correctly select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema describes the 'directory' parameter only as an optional string/null with no description. The tool description compensates by explaining that omitting the directory defaults to the configured inbox folder, adding meaningful semantic content beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List image files in a directory' with a specific verb and resource. It also distinguishes from sibling tools by positioning this as a prerequisite for describe_image, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises 'Use this before describe_image to find what is available,' giving a clear when-to-use instruction and a named alternative. This effectively communicates the intended workflow without needing to enumerate all siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_statusA

Show bridge configuration and Ollama connectivity / available models.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description indicates a read-only status operation via 'Show', but does not disclose potential error behaviors, such as what happens if Ollama is not reachable, nor does it confirm side-effect-free operation. With no annotations, a bit more detail would be helpful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that directly states the tool's purpose. Every word earns its place with no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (zero parameters, output schema present), the description adequately covers its purpose. However, it lacks any usage context or behavioral nuance (e.g., read-only nature, dependency on external services), which would make it fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is no parameter semantics to explain. The description adds value by indicating what the tool returns (configuration, connectivity, models), which is sufficient for a no-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Show') and resource ('bridge configuration and Ollama connectivity / available models'). This distinguishes it from sibling image processing tools like describe_image and list_images.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance is given for when to use this tool versus alternatives. The purpose implies a status check, but the description does not state any prerequisites or conditions (e.g., 'Use when checking Ollama availability').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observeddescribe_image
    • First observedextract_text
    • First observedlist_images
    • First observedvision_status
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.2/5.0

Scored across 4 tools

Disambiguation5/5

Each tool has a clear, distinct purpose: list_images finds images, describe_image provides semantic description, extract_text performs OCR, and vision_status checks configuration. There is no overlap or ambiguity between them.

Naming Consistency4/5

Tool names predominantly follow a verb_noun snake_case pattern (describe_image, list_images, extract_text). The exception is vision_status, which is noun_noun rather than verb_noun, but the inconsistency is minor and the naming style remains uniform.

Tool Count5/5

With 4 tools, the server is well-scoped for a focused vision MCP. Each tool serves a necessary role in the image-analysis workflow, and the count is neither too sparse nor excessive.

Completeness5/5

The tool surface covers the core workflows: discovering available images, getting a semantic description, extracting text, and verifying connectivity. There are no obvious gaps that would prevent an agent from completing typical vision-related tasks.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • ollama-vision-mcp

    ducnm9
    F
    license
    A
    quality
    D
    maintenance
    MCP server enabling LLM clients without vision capability to process images by delegating to local Ollama vision models. Supports describing images, OCR, asking questions, and processing clipboard images.
    4
    -

  • local-vision-mcp

    tmchao7
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for local Ollama vision analysis, enabling text-only agents like Claude Code to inspect images via a single tool. Processes images locally with Ollama, keeping image bytes on the machine and returning text reports.
    2
    MIT

  • fastcar-vision

    williamDazhangyu
    A
    license
    A
    quality
    B
    maintenance
    Provides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.
    3
    25 npm
    MIT