Skip to main content
Glama
deenesjakoruzh
masterLazy

ollama-vision-mcp

by masterLazy
OverviewSchemaRelated ServersScoreDiscussions
Python
Local

ollama-vision-mcp

English · 中文

A minimal bridging service that provides local vision capabilities for VS Code Copilot using text-only models (e.g., DeepSeek).

When Copilot uses a text-only model, it cannot directly "see" images. This MCP server fills that gap: you drop a screenshot into a folder; Copilot reads the image via this bridge → sends it to a local Ollama vision model → gets back a text description, which the text-only model can understand.

VS Code Copilot Chat (Agent mode, text-only model)
        │  MCP stdio
        ▼
ollama-vision-mcp (this package)
        │  Read local image → base64 → POST /v1/chat/completions
        ▼
Ollama (local vision model, e.g., qwen2.5vl:7b)

Tools

Tool

Parameters

Description

describe_image

path (required), mode?, question?

Reads an image and generates a text description. mode is optional: general (default), ocr, ui, diagram; question can provide an additional query.

list_images

directory? (default: inbox directory)

Lists image files available for reading.

extract_text

path (required)

Extracts text from the image via OCR.

vision_status

—

Displays the current configuration, Ollama connection status, and list of available models.

Related MCP server: Hybrid Vision MCP Server

Scope of This Tool

This package is only a bridge layer. It communicates with Ollama purely over HTTP (OpenAI-compatible /v1/chat/completions and /v1/models).

It does not handle:

  • Installing Ollama, starting ollama serve, pulling models, or managing model configuration;

  • Accessing the clipboard, IDE internal mechanisms, or any network service other than your local Ollama.

You are fully responsible for installing Ollama and pulling models (see Prerequisites).

Prerequisites

You must set up the following on your own:

  • Python 3.10+

  • Ollama installed and running: ollama serve

  • At least one vision model pulled; recommended:

    bash

    ollama pull qwen2.5vl:7b

    Other options: qwen3-vl:8b, gemma3:12b, llama3.2-vision:11b, llava:7b. Choose a model suitable for your GPU; specify it later via the VISION_MCP_MODEL environment variable.

🚀 Quick Install (Recommended)

Run a single command in the repository root to complete installation and configuration:

bash

cd ollama-vision-mcp
python setup_mcp.py

The script will automatically:

  1. Create a dedicated virtual environment (.venv) and install this package (without polluting the global environment).

  2. Detect the local Ollama service and list the available vision models.

  3. Interactively guide you to set base_url, model, inbox, max_tokens, image compression, and an optional API Key.

  4. Write the configuration to the project’s .vscode/mcp.json and/or the user‑level global MCP file (%APPDATA%\Code\User\mcp.json), merging with existing settings and not overwriting your other MCP servers.

Non-interactive usage (suitable for CI/scripts):

bash

python setup_mcp.py --yes --project --model qwen2.5vl:7b
python setup_mcp.py --print         # Only prints the config JSON, does not write to file

After installation, reload the window in VS Code (Ctrl+Shift+P → “Developer: Reload Window”) for the configuration to take effect.

If you have run it before, you can also reconfigure by simply using the command ollama-vision-setup.

Verify Installation

In VS Code Copilot Chat (Agent mode), enter:

text

Run vision_status

If the returned JSON contains "ollama": { "ok": true, ... } and the model list, the connection is successful. If ok is false, start Ollama (ollama serve) first and try again.

Usage

  1. Put screenshots into the project’s inbox directory (default .ai/inbox). The server will create this directory automatically when needed.

  2. Ask a question in Copilot Chat, for example:

    “Look at the screenshots in the inbox and tell me what this error is about.”

  3. The agent will automatically call list_images → describe_image and answer based on the text description.

Make the Agent Smarter (Optional): Copy the ready-made instruction file .github/instructions/ollama-vision/vision-tools.instructions.md into the same path in your project. It teaches the agent the full vision workflow — call order (list_images → describe_image → extract_text), mode selection, troubleshooting, and guiding you to drop screenshots into the inbox. As a VS Code file instruction it is discovered on-demand whenever the task involves images, so it works out of the box with no setup.

Environment Variables Reference

Variable

Default

Description

VISION_MCP_BASE_URL

http://localhost:11434/v1

Ollama’s OpenAI-compatible base URL

VISION_MCP_MODEL

qwen2.5vl:7b

Vision model to use

VISION_MCP_API_KEY

empty (actually uses ollama)

API Key (ignored by Ollama)

VISION_MCP_INBOX

.ai/inbox

Default directory for list_images

VISION_MCP_MAX_TOKENS

2048

Maximum output tokens for the vision model

VISION_MCP_COMPRESS

1

If image > 50KB, automatically resize to 768px JPEG

VISION_MCP_THINK

0

Keep thinking enabled for thinking models (1), or disable it via reasoning_effort=none (0, default)

Compatibility aliases: VISION_MODEL, VISION_BASE_URL, VISION_INBOX, VISION_API_KEY.

Manual Install (Alternative)

If you need full manual control over each step, refer to the process below.

1. Create an environment and install

bash

cd ollama-vision-mcp
python -m venv .venv
.venv\Scripts\activate          # Windows
# source .venv/bin/activate    # macOS / Linux
pip install -e .

After installation, you get two commands:

  • ollama-vision-mcp — start the MCP server

  • ollama-vision-setup — interactive configuration shortcut

2. Register with VS Code

Copy the following into your project’s .vscode/mcp.json (project only) or the user-level file %APPDATA%\Code\User\mcp.json (all projects). Make sure to change command to the absolute path of your virtual environment’s Python interpreter:

json

{
  "servers": {
    "ollama-vision": {
      "type": "stdio",
      "command": "D:/MyRepos/ollama-vision-mcp/.venv/Scripts/python.exe",
      "args": ["-m", "ollama_vision_mcp"],
      "env": {
        "VISION_MCP_BASE_URL": "http://localhost:11434/v1",
        "VISION_MCP_MODEL": "qwen2.5vl:7b",
        "VISION_MCP_INBOX": ".ai/inbox"
      }
    }
  }
}

⚠️ command must be the absolute path to the Python interpreter inside the virtual environment. You can also run ollama-vision-setup --project to generate this file automatically. The server uses os.getcwd() when invoked, so relative paths (like path and the inbox directory) will be resolved relative to the project root where the server is started.

Smoke Test

Run the following script to check Ollama connectivity, list the inbox contents, and perform a real vision call on the first image:

bash

python examples/smoke_test.py

License

MIT

Available Tools

4 tools
describe_imageA

Read a local image and return a text description from the local Ollama vision model.

Use when the user references a screenshot or image you cannot see. path is absolute, or relative to the project directory.

ParametersJSON Schema
NameRequiredDescriptionDefault
modeNogeneral
pathYes
questionNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the local Ollama model and path semantics, but does not explain behavior around missing files, model limitations, or the purpose of mode/question parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose, followed by usage context and path semantics. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although an output schema exists, the description omits crucial parameter semantics for `mode` and `question`, which directly affect tool behavior. Without these details, an agent cannot fully understand how to use the tool for specialized cases like OCR or diagram analysis.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It explains only the `path` parameter ('absolute, or relative to the project directory'), leaving `mode` and `question` completely unexplained. This is a significant gap for a tool with three parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Read a local image and return a text description from the local Ollama vision model.' This specifies the action, resource, and method, and distinguishes it from siblings like extract_text by focusing on generating a description rather than extracting text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use when the user references a screenshot or image you cannot see.' This gives a clear condition for use, though it does not explicitly mention alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_textA

Extract all visible text (OCR) from a local image verbatim.

Use for code screenshots, terminal output, or error dialogs.

ParametersJSON Schema
NameRequiredDescriptionDefault
pathYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. It states that OCR is used, that all visible text is extracted, and that output is verbatim, which are meaningful behavioral traits. It does not cover failure modes or image format limitations, but the core behavior is clearly disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no redundancy. The first sentence states the action and scope, and the second lists concrete use cases. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with an output schema and no complex side effects, the description covers the essential operational context: what it does, when to use it, and the key behavioral constraint (verbatim OCR). It is not exhaustive but adequately complete for its low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the only parameter is 'path' and the description's phrase 'from a local image' clearly establishes that path refers to a local image file. This compensates for the lack of schema documentation, though it does not specify path syntax or accepted formats.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Extract all visible text (OCR) from a local image verbatim.' It clearly distinguishes this tool from siblings like describe_image by focusing on text extraction rather than image description, and adds qualifiers ('all visible', 'verbatim') that make the scope precise.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives concrete use cases: 'Use for code screenshots, terminal output, or error dialogs.' This provides clear context for when to use the tool, though it does not explicitly mention alternatives or exclusions relative to sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_imagesA

List image files in a directory.

Defaults to the configured inbox folder where the user drops screenshots. Use this before describe_image to find what is available.

ParametersJSON Schema
NameRequiredDescriptionDefault
directoryNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry behavioral transparency. It discloses the default behavior of falling back to the configured inbox folder, which is valuable. However, it does not specify whether subdirectories are included, result sorting, or other behavioral traits, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise, front-loaded sentences. Every word adds value, with no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple listing tool with an output schema present, the description covers purpose, default behavior, and usage guidance. It is sufficiently complete for an agent to correctly select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema describes the 'directory' parameter only as an optional string/null with no description. The tool description compensates by explaining that omitting the directory defaults to the configured inbox folder, adding meaningful semantic content beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List image files in a directory' with a specific verb and resource. It also distinguishes from sibling tools by positioning this as a prerequisite for describe_image, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises 'Use this before describe_image to find what is available,' giving a clear when-to-use instruction and a named alternative. This effectively communicates the intended workflow without needing to enumerate all siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_statusA

Show bridge configuration and Ollama connectivity / available models.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description indicates a read-only status operation via 'Show', but does not disclose potential error behaviors, such as what happens if Ollama is not reachable, nor does it confirm side-effect-free operation. With no annotations, a bit more detail would be helpful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that directly states the tool's purpose. Every word earns its place with no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (zero parameters, output schema present), the description adequately covers its purpose. However, it lacks any usage context or behavioral nuance (e.g., read-only nature, dependency on external services), which would make it fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is no parameter semantics to explain. The description adds value by indicating what the tool returns (configuration, connectivity, models), which is sufficient for a no-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Show') and resource ('bridge configuration and Ollama connectivity / available models'). This distinguishes it from sibling image processing tools like describe_image and list_images.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance is given for when to use this tool versus alternatives. The purpose implies a status check, but the description does not state any prerequisites or conditions (e.g., 'Use when checking Ollama availability').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 4 tool updatesv0.1.0
    • First observeddescribe_image
    • First observedextract_text
    • First observedlist_images
    • First observedvision_status
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.2/5.0
Disambiguation5/5

Each tool has a clear, distinct purpose: list_images finds images, describe_image provides semantic description, extract_text performs OCR, and vision_status checks configuration. There is no overlap or ambiguity between them.

Naming Consistency4/5

Tool names predominantly follow a verb_noun snake_case pattern (describe_image, list_images, extract_text). The exception is vision_status, which is noun_noun rather than verb_noun, but the inconsistency is minor and the naming style remains uniform.

Tool Count5/5

With 4 tools, the server is well-scoped for a focused vision MCP. Each tool serves a necessary role in the image-analysis workflow, and the count is neither too sparse nor excessive.

Completeness5/5

The tool surface covers the core workflows: discovering available images, getting a semantic description, extracting text, and verifying connectivity. There are no obvious gaps that would prevent an agent from completing typical vision-related tasks.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • ollama-vision-mcp

    ducnm9
    F
    license
    A
    quality
    C
    maintenance
    MCP server enabling LLM clients without vision capability to process images by delegating to local Ollama vision models. Supports describing images, OCR, asking questions, and processing clipboard images.
    4
    -

  • local-vision-mcp

    tmchao7
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for local Ollama vision analysis, enabling text-only agents like Claude Code to inspect images via a single tool. Processes images locally with Ollama, keeping image bytes on the machine and returning text reports.
    2
    MIT

  • fastcar-vision

    williamDazhangyu
    A
    license
    A
    quality
    B
    maintenance
    Provides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.
    3
    174
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/masterLazy/ollama-vision-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server