Skip to main content
Glama
deenesjakoruzh
masterLazy

ollama-vision-mcp

by masterLazy
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

ollama-vision-mcp

English · 中文

Минимальный мостовой сервис, который предоставляет локальные vision-возможности для VS Code Copilot, использующего текстовые модели (например, DeepSeek).

Когда Copilot использует текстовую модель, он не может напрямую «видеть» изображения. Этот MCP-сервер устраняет этот пробел: вы кладёте скриншот в папку; Copilot читает изображение через этот мост → отправляет его в локальную Ollama vision-модель → получает обратно текстовое описание, которое текстовая модель может понять.

VS Code Copilot Chat (Agent mode, text-only model)
        │  MCP stdio
        ▼
ollama-vision-mcp (this package)
        │  Read local image → base64 → POST /v1/chat/completions
        ▼
Ollama (local vision model, e.g., qwen2.5vl:7b)

Инструменты

Инструмент

Параметры

Описание

describe_image

path (required), mode?, question?

Читает изображение и генерирует текстовое описание. mode необязателен: general (по умолчанию), ocr, ui, diagram; question может содержать дополнительный запрос.

list_images

directory? (по умолчанию: каталог inbox)

Перечисляет файлы изображений, доступные для чтения.

extract_text

path (required)

Извлекает текст из изображения с помощью OCR.

vision_status

—

Отображает текущую конфигурацию, статус подключения к Ollama и список доступных моделей.

Related MCP server: Hybrid Vision MCP Server

Область применения этого инструмента

Этот пакет — только мостовой слой. Он взаимодействует с Ollama исключительно по HTTP (совместимый с OpenAI /v1/chat/completions и /v1/models).

Он не обрабатывает:

  • Установку Ollama, запуск ollama serve, загрузку моделей или управление конфигурацией моделей;

  • Доступ к буферу обмена, внутренним механизмам IDE или любым сетевым сервисам, кроме вашего локального Ollama.

Вы полностью отвечаете за установку Ollama и загрузку моделей (см. раздел «Предварительные требования»).

Предварительные требования

Вы должны самостоятельно настроить следующее:

  • Python 3.10+

  • Ollama установлен и запущен: ollama serve

  • Хотя бы одна vision-модель загружена; рекомендуется:

    bash

    ollama pull qwen2.5vl:7b

    Другие варианты: qwen3-vl:8b, gemma3:12b, llama3.2-vision:11b, llava:7b. Выберите модель, подходящую для вашего GPU; укажите её позже через переменную окружения VISION_MCP_MODEL.

🚀 Быстрая установка (рекомендуется)

Выполните одну команду в корне репозитория, чтобы завершить установку и настройку:

bash GXP3

Скрипт автоматически:

  1. Создаст выделенное виртуальное окружение (.venv) и установит этот пакет (не затрагивая глобальное окружение).

  2. Обнаружит локальный сервис Ollama и выведет список доступных vision-моделей.

  3. Интерактивно проведёт вас по настройке base_url, model, inbox, max_tokens, сжатия изображений и необязательного API Key.

  4. Запишет конфигурацию в .vscode/mcp.json проекта и/или в глобальный MCP-файл уровня пользователя (%APPDATA%\Code\User\mcp.json), объединяя с существующими настройками и не перезаписывая ваши другие MCP-серверы.

Неинтерактивное использование (подходит для CI/скриптов):

bash GXP4

После установки перезагрузите окно в VS Code (Ctrl+Shift+P → «Developer: Reload Window»), чтобы конфигурация вступила в силу.

Если вы уже запускали его ранее, вы также можете перенастроить всё, просто выполнив команду ollama-vision-setup.

Проверка установки

В VS Code Copilot Chat (в режиме агента) введите:

text GXP5

Если возвращённый JSON содержит "ollama": { "ok": true, ... } и список моделей, соединение успешно. Если ok имеет значение false, сначала запустите Ollama (ollama serve) и попробуйте снова.

Использование

  1. Поместите скриншоты в каталог inbox проекта (по умолчанию .ai/inbox). Сервер создаст этот каталог автоматически при необходимости.

  2. Задайте вопрос в Copilot Chat, например:

    «Посмотри на скриншоты в inbox и скажи, о чём эта ошибка».

  3. Агент автоматически вызовет list_images → describe_image и ответит на основе текстового описания.

Сделайте агента умнее (необязательно): скопируйте готовый файл инструкций .github/instructions/ollama-vision/vision-tools.instructions.md в тот же путь в вашем проекте. Он обучает агента полному рабочему процессу vision — порядку вызовов (list_images → describe_image → extract_text), выбору режима, устранению неполадок и подсказывает, куда класть скриншоты (в inbox). Будучи файловой инструкцией VS Code, он обнаруживается по запросу всякий раз, когда задача связана с изображениями, поэтому работает сразу без настройки.

Справочник переменных окружения

Переменная

По умолчанию

Описание

VISION_MCP_BASE_URL

http://localhost:11434/v1

Базовый URL Ollama, совместимый с OpenAI

VISION_MCP_MODEL

qwen2.5vl:7b

Vision-модель для использования

VISION_MCP_API_KEY

пусто (фактически используется ollama)

API Key (игнорируется Ollama)

VISION_MCP_INBOX

.ai/inbox

Каталог по умолчанию для list_images

VISION_MCP_MAX_TOKENS

2048

Максимальное количество выходных токенов для vision-модели

VISION_MCP_COMPRESS

1

Если изображение > 50 КБ, автоматически изменить размер до JPEG 768 px

VISION_MCP_THINK

0

Оставить режим размышлений включённым для думающих моделей (1) или отключить его через reasoning_effort=none (0, по умолчанию)

Псевдонимы совместимости: VISION_MODEL, VISION_BASE_URL, VISION_INBOX, VISION_API_KEY.

Установка вручную (альтернативный вариант)

Если вам нужен полный ручной контроль над каждым шагом, следуйте приведённому ниже процессу.

1. Создайте окружение и установите

bash GXP6

После установки у вас появятся две команды:

  • ollama-vision-mcp — запуск MCP-сервера

  • ollama-vision-setup — интерактивный ярлык настройки

2. Зарегистрируйте в VS Code

Скопируйте следующее в .vscode/mcp.json вашего проекта (только для проекта) или в пользовательский файл %APPDATA%\Code\User\mcp.json (для всех проектов). Обязательно измените command на абсолютный путь к интерпретатору Python в вашем виртуальном окружении:

json GXP7

⚠️ command должен быть абсолютным путём к интерпретатору Python внутри виртуального окружения. Вы также можете выполнить ollama-vision-setup --project, чтобы сгенерировать этот файл автоматически. При вызове сервер использует os.getcwd(), поэтому относительные пути (например, path и каталог inbox) будут разрешаться относительно корня проекта, в котором запущен сервер.

Дымовой тест

Выполните следующий скрипт, чтобы проверить подключение к Ollama, вывести содержимое inbox и выполнить реальный vision-вызов на первом изображении:

bash GXP8

Лицензия

MIT

Available Tools

4 tools
describe_imageA

Read a local image and return a text description from the local Ollama vision model.

Use when the user references a screenshot or image you cannot see. path is absolute, or relative to the project directory.

NameRequiredDescriptionDefault
modeNogeneral
pathYes
questionNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the local Ollama model and path semantics, but does not explain behavior around missing files, model limitations, or the purpose of mode/question parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences, front-loaded with the core purpose, followed by usage context and path semantics. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although an output schema exists, the description omits crucial parameter semantics for `mode` and `question`, which directly affect tool behavior. Without these details, an agent cannot fully understand how to use the tool for specialized cases like OCR or diagram analysis.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It explains only the `path` parameter ('absolute, or relative to the project directory'), leaving `mode` and `question` completely unexplained. This is a significant gap for a tool with three parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Read a local image and return a text description from the local Ollama vision model.' This specifies the action, resource, and method, and distinguishes it from siblings like extract_text by focusing on generating a description rather than extracting text.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance: 'Use when the user references a screenshot or image you cannot see.' This gives a clear condition for use, though it does not explicitly mention alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

extract_textA

Extract all visible text (OCR) from a local image verbatim.

Use for code screenshots, terminal output, or error dialogs.

NameRequiredDescriptionDefault
pathYes

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. It states that OCR is used, that all visible text is extracted, and that output is verbatim, which are meaningful behavioral traits. It does not cover failure modes or image format limitations, but the core behavior is clearly disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no redundancy. The first sentence states the action and scope, and the second lists concrete use cases. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with an output schema and no complex side effects, the description covers the essential operational context: what it does, when to use it, and the key behavioral constraint (verbatim OCR). It is not exhaustive but adequately complete for its low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the only parameter is 'path' and the description's phrase 'from a local image' clearly establishes that path refers to a local image file. This compensates for the lack of schema documentation, though it does not specify path syntax or accepted formats.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a specific verb and resource: 'Extract all visible text (OCR) from a local image verbatim.' It clearly distinguishes this tool from siblings like describe_image by focusing on text extraction rather than image description, and adds qualifiers ('all visible', 'verbatim') that make the scope precise.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives concrete use cases: 'Use for code screenshots, terminal output, or error dialogs.' This provides clear context for when to use the tool, though it does not explicitly mention alternatives or exclusions relative to sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_imagesA

List image files in a directory.

Defaults to the configured inbox folder where the user drops screenshots. Use this before describe_image to find what is available.

NameRequiredDescriptionDefault
directoryNo

Output Schema

NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry behavioral transparency. It discloses the default behavior of falling back to the configured inbox folder, which is valuable. However, it does not specify whether subdirectories are included, result sorting, or other behavioral traits, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise, front-loaded sentences. Every word adds value, with no fluff or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple listing tool with an output schema present, the description covers purpose, default behavior, and usage guidance. It is sufficiently complete for an agent to correctly select and invoke the tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema describes the 'directory' parameter only as an optional string/null with no description. The tool description compensates by explaining that omitting the directory defaults to the configured inbox folder, adding meaningful semantic content beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List image files in a directory' with a specific verb and resource. It also distinguishes from sibling tools by positioning this as a prerequisite for describe_image, making its purpose unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly advises 'Use this before describe_image to find what is available,' giving a clear when-to-use instruction and a named alternative. This effectively communicates the intended workflow without needing to enumerate all siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

vision_statusA

Show bridge configuration and Ollama connectivity / available models.

NameRequiredDescriptionDefault

No parameters

Output Schema

NameRequiredDescription
resultYes

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description indicates a read-only status operation via 'Show', but does not disclose potential error behaviors, such as what happens if Ollama is not reachable, nor does it confirm side-effect-free operation. With no annotations, a bit more detail would be helpful.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that directly states the tool's purpose. Every word earns its place with no fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (zero parameters, output schema present), the description adequately covers its purpose. However, it lacks any usage context or behavioral nuance (e.g., read-only nature, dependency on external services), which would make it fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is no parameter semantics to explain. The description adds value by indicating what the tool returns (configuration, connectivity, models), which is sufficient for a no-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's function with a specific verb ('Show') and resource ('bridge configuration and Ollama connectivity / available models'). This distinguishes it from sibling image processing tools like describe_image and list_images.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance is given for when to use this tool versus alternatives. The purpose implies a status check, but the description does not state any prerequisites or conditions (e.g., 'Use when checking Ollama availability').

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observeddescribe_image
    • First observedextract_text
    • First observedlist_images
    • First observedvision_status
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A4.2/5.0

Scored across 4 tools

Disambiguation5/5

Each tool has a clear, distinct purpose: list_images finds images, describe_image provides semantic description, extract_text performs OCR, and vision_status checks configuration. There is no overlap or ambiguity between them.

Naming Consistency4/5

Tool names predominantly follow a verb_noun snake_case pattern (describe_image, list_images, extract_text). The exception is vision_status, which is noun_noun rather than verb_noun, but the inconsistency is minor and the naming style remains uniform.

Tool Count5/5

With 4 tools, the server is well-scoped for a focused vision MCP. Each tool serves a necessary role in the image-analysis workflow, and the count is neither too sparse nor excessive.

Completeness5/5

The tool surface covers the core workflows: discovering available images, getting a semantic description, extracting text, and verifying connectivity. There are no obvious gaps that would prevent an agent from completing typical vision-related tasks.

Maintenance

ActivityStale
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • ollama-vision-mcp

    ducnm9
    F
    license
    A
    quality
    D
    maintenance
    MCP server enabling LLM clients without vision capability to process images by delegating to local Ollama vision models. Supports describing images, OCR, asking questions, and processing clipboard images.
    4
    -

  • local-vision-mcp

    tmchao7
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for local Ollama vision analysis, enabling text-only agents like Claude Code to inspect images via a single tool. Processes images locally with Ollama, keeping image bytes on the machine and returning text reports.
    2
    MIT

  • fastcar-vision

    williamDazhangyu
    A
    license
    A
    quality
    B
    maintenance
    Provides vision understanding capabilities such as image analysis, OCR, object localization, and video frame analysis, plus optional image generation and editing, to coding agents via OpenAI-compatible multimodal models. Runs as a local MCP server with HTTP and stdio transports, configurable for clients like Codex, Claude Code, Kimi, and Cursor.
    3
    25 npm
    MIT