Skip to main content
Glama
Sign In
deenesjakoruzh
ollama-vision-mcp

ollama-vision-mcp

by masterLazy
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

ollama-vision-mcp

Englisch · 中文

Ein minimaler Brückendienst, der lokale Bildverarbeitungsfunktionen für VS Code Copilot mit reinen Textmodellen (z. B. DeepSeek) bereitstellt.

Wenn Copilot ein reines Textmodell verwendet, kann es Bilder nicht direkt „sehen“. Dieser MCP-Server schließt diese Lücke: Sie legen einen Screenshot in einen Ordner; Copilot liest das Bild über diese Brücke → sendet es an ein lokales Ollama-Vision-Modell → erhält eine Textbeschreibung zurück, die das reine Textmodell verstehen kann.

VS Code Copilot Chat (Agent mode, text-only model)
        │  MCP stdio
        ▼
ollama-vision-mcp (this package)
        │  Read local image → base64 → POST /v1/chat/completions
        ▼
Ollama (local vision model, e.g., qwen2.5vl:7b)

Tools

Tool

Parameter

Beschreibung

describe_image

path (erforderlich), mode?, question?

Liest ein Bild und erzeugt eine Textbeschreibung. mode ist optional: general (Standard), ocr, ui, diagram; question kann eine zusätzliche Abfrage bereitstellen.

list_images

directory? (Standard: Inbox-Verzeichnis)

Listet Bilddateien auf, die zum Lesen verfügbar sind.

extract_text

path (erforderlich)

Extrahiert Text aus dem Bild per OCR.

vision_status

—

Zeigt die aktuelle Konfiguration, den Ollama-Verbindungsstatus und die Liste der verfügbaren Modelle an.

Related MCP server: Hybrid Vision MCP Server

Umfang dieses Tools

Dieses Paket ist nur eine Brückenschicht. Es kommuniziert mit Ollama ausschließlich über HTTP (OpenAI-kompatible /v1/chat/completions und /v1/models).

Es übernimmt nicht:

  • Installation von Ollama, Start von ollama serve, Herunterladen von Modellen oder Verwaltung der Modellkonfiguration;

  • Zugriff auf die Zwischenablage, interne IDE-Mechanismen oder andere Netzwerkdienste als Ihr lokales Ollama.

Sie sind vollständig für die Installation von Ollama und das Herunterladen von Modellen verantwortlich (siehe Voraussetzungen).

Voraussetzungen

Sie müssen Folgendes selbst einrichten:

  • Python 3.10+

  • Ollama installiert und ausgeführt: ollama serve

  • Mindestens ein Vision-Modell heruntergeladen; empfohlen:

    bash

    ollama pull qwen2.5vl:7b

    Weitere Optionen: qwen3-vl:8b, gemma3:12b, llama3.2-vision:11b, llava:7b. Wählen Sie ein für Ihre GPU geeignetes Modell; geben Sie es später über die Umgebungsvariable VISION_MCP_MODEL an.

🚀 Schnellinstallation (Empfohlen)

Führen Sie einen einzigen Befehl im Repository-Stammverzeichnis aus, um Installation und Konfiguration abzuschließen:

bash

cd ollama-vision-mcp
python setup_mcp.py

Das Skript wird automatisch:

  1. Eine dedizierte virtuelle Umgebung (.venv) erstellen und dieses Paket installieren (ohne die globale Umgebung zu beeinträchtigen).

  2. Den lokalen Ollama-Dienst erkennen und die verfügbaren Vision-Modelle auflisten.

  3. Sie interaktiv durch die Einstellung von base_url, model, inbox, max_tokens, Bildkomprimierung und einem optionalen API-Schlüssel führen.

  4. Die Konfiguration in die Projektdatei .vscode/mcp.json und/oder die benutzerweite globale MCP-Datei (%APPDATA%\Code\User\mcp.json) schreiben, zusammengeführt mit vorhandenen Einstellungen, ohne Ihre anderen MCP-Server zu überschreiben.

Nicht-interaktive Nutzung (geeignet für CI/Skripte):

bash

python setup_mcp.py --yes --project --model qwen2.5vl:7b
python setup_mcp.py --print         # Only prints the config JSON, does not write to file

Nach der Installation laden Sie das Fenster in VS Code neu (Strg+Umschalt+P → „Developer: Reload Window“), damit die Konfiguration wirksam wird.

Wenn Sie es bereits ausgeführt haben, können Sie es auch einfach mit dem Befehl ollama-vision-setup neu konfigurieren.

Installation überprüfen

Geben Sie in VS Code Copilot Chat (Agent-Modus) ein:

text

Run vision_status

Wenn das zurückgegebene JSON "ollama": { "ok": true, ... } und die Modellliste enthält, ist die Verbindung erfolgreich. Wenn ok falsch ist, starten Sie zuerst Ollama (ollama serve) und versuchen Sie es erneut.

Verwendung

  1. Legen Sie Screenshots in das Inbox-Verzeichnis des Projekts (Standard: .ai/inbox). Der Server erstellt dieses Verzeichnis bei Bedarf automatisch.

  2. Stellen Sie eine Frage in Copilot Chat, zum Beispiel:

    „Schauen Sie sich die Screenshots im Posteingang an und sagen Sie mir, worum es bei diesem Fehler geht.“

  3. Der Agent ruft automatisch list_images → describe_image auf und antwortet basierend auf der Textbeschreibung.

Den Agenten intelligenter machen (optional): Kopieren Sie die vorgefertigte Anweisungsdatei .github/instructions/ollama-vision/vision-tools.instructions.md in denselben Pfad in Ihrem Projekt. Sie vermittelt dem Agenten den vollständigen Vision-Workflow – Aufrufreihenfolge (list_images → describe_image → extract_text), Modusauswahl, Fehlerbehebung und Anleitung, Screenshots in den Posteingang zu legen. Als VS-Code-Dateianweisung wird sie bei Bedarf immer dann erkannt, wenn die Aufgabe Bilder betrifft, und funktioniert daher ohne Einrichtung sofort.

Referenz der Umgebungsvariablen

Variable

Standard

Beschreibung

VISION_MCP_BASE_URL

http://localhost:11434/v1

OpenAI-kompatible Basis-URL von Ollama

VISION_MCP_MODEL

qwen2.5vl:7b

Zu verwendendes Vision-Modell

VISION_MCP_API_KEY

leer (verwendet tatsächlich ollama)

API-Schlüssel (von Ollama ignoriert)

VISION_MCP_INBOX

.ai/inbox

Standardverzeichnis für list_images

VISION_MCP_MAX_TOKENS

2048

Maximale Ausgabetokens für das Vision-Modell

VISION_MCP_COMPRESS

1

Wenn Bild > 50 KB, automatisch auf 768px JPEG skalieren

VISION_MCP_THINK

0

Denken für Denkmodelle aktiviert lassen (1) oder über reasoning_effort=none deaktivieren (0, Standard)

Kompatibilitäts-Aliasse: VISION_MODEL, VISION_BASE_URL, VISION_INBOX, VISION_API_KEY

Manuelle Installation (Alternative)

Wenn Sie die volle manuelle Kontrolle über jeden Schritt benötigen, beachten Sie den folgenden Ablauf.

1. Umgebung erstellen und installieren

bash

cd ollama-vision-mcp
python -m venv .venv
.venv\Scripts\activate          # Windows
# source .venv/bin/activate    # macOS / Linux
pip install -e .

Nach der Installation erhalten Sie zwei Befehle:

  • ollama-vision-mcp — startet den MCP-Server

  • ollama-vision-setup — interaktive Konfigurationsabkürzung

2. Bei VS Code registrieren

Kopieren Sie Folgendes in die .vscode/mcp.json Ihres Projekts (nur Projekt) oder in die benutzerweite Datei %APPDATA%\Code\User\mcp.json (alle Projekte). Stellen Sie sicher, dass Sie command auf den absoluten Pfad des Python-Interpreters Ihrer virtuellen Umgebung ändern:

json

{
  "servers": {
    "ollama-vision": {
      "type": "stdio",
      "command": "D:/MyRepos/ollama-vision-mcp/.venv/Scripts/python.exe",
      "args": ["-m", "ollama_vision_mcp"],
      "env": {
        "VISION_MCP_BASE_URL": "http://localhost:11434/v1",
        "VISION_MCP_MODEL": "qwen2.5vl:7b",
        "VISION_MCP_INBOX": ".ai/inbox"
      }
    }
  }
}

⚠️ command muss der absolute Pfad zum Python-Interpreter innerhalb der virtuellen Umgebung sein. Sie können auch ollama-vision-setup --project ausführen, um diese Datei automatisch zu generieren. Der Server verwendet beim Aufruf os.getcwd(), daher werden relative Pfade (wie path und das Inbox-Verzeichnis) relativ zum Projektstamm aufgelöst, in dem der Server gestartet wird.

Rauchtest

Führen Sie das folgende Skript aus, um die Ollama-Konnektivität zu überprüfen, den Inhalt des Posteingangs aufzulisten und einen echten Vision-Aufruf für das erste Bild durchzuführen:

bash

python examples/smoke_test.py

Lizenz

MIT

Available Tools

4 tools
describe_imageA

Read a local image and return a text description from the local Ollama vision model.

Use when the user references a screenshot or image you cannot see. path is absolute, or relative to the project directory.

NameRequiredDescriptionDefault
modeNogeneral
pathYes
questionNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the local Ollama model and path semantics, but does not explain behavior around missing files, model limitations, or the purpose of mode/question parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose, followed by usage context and path semantics. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although an output schema exists, the description omits crucial parameter semantics for `mode` and `question`, which directly affect tool behavior. Without these details, an agent cannot fully understand how to use the tool for specialized cases like OCR or diagram analysis.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It explains only the `path` parameter ('absolute, or relative to the project directory'), leaving `mode` and `question` completely unexplained. This is a significant gap for a tool with three parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Read a local image and return a text description from the local Ollama vision model.' This specifies the action, resource, and method, and distinguishes it from siblings like extract_text by focusing on generating a description rather than extracting text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use when the user references a screenshot or image you cannot see.' This gives a clear condition for use, though it does not explicitly mention alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_textA

Extract all visible text (OCR) from a local image verbatim.

Use for code screenshots, terminal output, or error dialogs.

NameRequiredDescriptionDefault
pathYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. It states that OCR is used, that all visible text is extracted, and that output is verbatim, which are meaningful behavioral traits. It does not cover failure modes or image format limitations, but the core behavior is clearly disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no redundancy. The first sentence states the action and scope, and the second lists concrete use cases. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with an output schema and no complex side effects, the description covers the essential operational context: what it does, when to use it, and the key behavioral constraint (verbatim OCR). It is not exhaustive but adequately complete for its low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the only parameter is 'path' and the description's phrase 'from a local image' clearly establishes that path refers to a local image file. This compensates for the lack of schema documentation, though it does not specify path syntax or accepted formats.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Extract all visible text (OCR) from a local image verbatim.' It clearly distinguishes this tool from siblings like describe_image by focusing on text extraction rather than image description, and adds qualifiers ('all visible', 'verbatim') that make the scope precise.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives concrete use cases: 'Use for code screenshots, terminal output, or error dialogs.' This provides clear context for when to use the tool, though it does not explicitly mention alternatives or exclusions relative to sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_imagesA

List image files in a directory.

Defaults to the configured inbox folder where the user drops screenshots. Use this before describe_image to find what is available.

NameRequiredDescriptionDefault
directoryNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry behavioral transparency. It discloses the default behavior of falling back to the configured inbox folder, which is valuable. However, it does not specify whether subdirectories are included, result sorting, or other behavioral traits, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise, front-loaded sentences. Every word adds value, with no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple listing tool with an output schema present, the description covers purpose, default behavior, and usage guidance. It is sufficiently complete for an agent to correctly select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema describes the 'directory' parameter only as an optional string/null with no description. The tool description compensates by explaining that omitting the directory defaults to the configured inbox folder, adding meaningful semantic content beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List image files in a directory' with a specific verb and resource. It also distinguishes from sibling tools by positioning this as a prerequisite for describe_image, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises 'Use this before describe_image to find what is available,' giving a clear when-to-use instruction and a named alternative. This effectively communicates the intended workflow without needing to enumerate all siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_statusA

Show bridge configuration and Ollama connectivity / available models.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description indicates a read-only status operation via 'Show', but does not disclose potential error behaviors, such as what happens if Ollama is not reachable, nor does it confirm side-effect-free operation. With no annotations, a bit more detail would be helpful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that directly states the tool's purpose. Every word earns its place with no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (zero parameters, output schema present), the description adequately covers its purpose. However, it lacks any usage context or behavioral nuance (e.g., read-only nature, dependency on external services), which would make it fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is no parameter semantics to explain. The description adds value by indicating what the tool returns (configuration, connectivity, models), which is sufficient for a no-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Show') and resource ('bridge configuration and Ollama connectivity / available models'). This distinguishes it from sibling image processing tools like describe_image and list_images.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance is given for when to use this tool versus alternatives. The purpose implies a status check, but the description does not state any prerequisites or conditions (e.g., 'Use when checking Ollama availability').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observeddescribe_image
    • First observedextract_text
    • First observedlist_images
    • First observedvision_status
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.2/5.0

Scored across 4 tools

Disambiguation5/5

Each tool has a clear, distinct purpose: list_images finds images, describe_image provides semantic description, extract_text performs OCR, and vision_status checks configuration. There is no overlap or ambiguity between them.

Naming Consistency4/5

Tool names predominantly follow a verb_noun snake_case pattern (describe_image, list_images, extract_text). The exception is vision_status, which is noun_noun rather than verb_noun, but the inconsistency is minor and the naming style remains uniform.

Tool Count5/5

With 4 tools, the server is well-scoped for a focused vision MCP. Each tool serves a necessary role in the image-analysis workflow, and the count is neither too sparse nor excessive.

Completeness5/5

The tool surface covers the core workflows: discovering available images, getting a semantic description, extracting text, and verifying connectivity. There are no obvious gaps that would prevent an agent from completing typical vision-related tasks.

Maintenance

ActivityInactive
ResponsivenessNo recent community issues

Resources

Related MCP Connectors

Related MCP Servers

  • ollama-vision-mcp

    ducnm9
    F
    license
    A
    quality
    D
    maintenance
    MCP server enabling LLM clients without vision capability to process images by delegating to local Ollama vision models. Supports describing images, OCR, asking questions, and processing clipboard images.
    4
    -

  • local-vision-mcp

    tmchao7
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for local Ollama vision analysis, enabling text-only agents like Claude Code to inspect images via a single tool. Processes images locally with Ollama, keeping image bytes on the machine and returning text reports.
    2
    MIT

  • fastcar-vision

    williamDazhangyu
    A
    license
    A
    quality
    C
    maintenance
    Provides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.
    3
    25 npm
    MIT