Which integrations are available for this server?

Integrates with Electron CDP to capture screenshots and perform vision analysis on Electron applications. Uses local Ollama models for AI-powered vision analysis, including image description, OCR, and text detection. Allows analyzing screenshots from Selenium-driven browser automation using vision capabilities.

How do I use vision-mcp?

1. Click on "Deploy Server". 2. Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state. 3. In the chat, type @ followed by the MCP server name and your instructions, e.g., "@vision-mcp analyze screenshot.png for error messages" That's it! The server will respond to your query, and you can continue using it as needed. Here is a step-by-step guide with screenshots.

vision-mcp

by arealicehole

Overview Schema Source Related Servers Discussions Admin

JavaScript

Local

Vision MCP Server 🔍

A powerful Model Context Protocol (MCP) server that provides AI-powered vision analysis capabilities using local Ollama models. Analyze screenshots, extract text, detect UI elements, and debug applications with state-of-the-art vision language models.

License: MIT Node.js Ollama

🚀 Features

🔍 Vision Analysis - Analyze screenshots and describe UI state, errors, or issues
📝 OCR Extraction - Extract text from images using VLM or Tesseract
🎯 Text Detection - Find specific text with bounding boxes for automation
🏥 Health Monitoring - Check Ollama connection and available models
🔌 Universal Integration - Works with Electron, Selenium, Playwright, and more
🏃‍♂️ CLI Tool - Standalone command-line interface for any workflow
⚡ High Performance - Optimized for 16GB+ VRAM with local models

Related MCP server: npu-vision-fallback

📦 Installation

Prerequisites

Node.js 18+
Ollama (for vision models)
Claude Code (for MCP integration)
16GB+ VRAM recommended for optimal performance

Quick Setup

# 1. Clone the repository
git clone https://github.com/YOUR_USERNAME/vision-mcp.git
cd vision-mcp

# 2. Install dependencies
npm install

# 3. Install Ollama and models
curl -fsSL https://ollama.com/install.sh | sh
ollama serve &
ollama pull llava:7b

# 4. Run setup script
chmod +x setup.sh
./setup.sh

# 5. Test installation
./vlm.mjs health

Claude Code Integration

Add to your ~/.claude.json in the mcpServers section:

{
  "mcpServers": {
    "vision": {
      "type": "stdio",
      "command": "/home/ice/vision-mcp-wrapper.sh",
      "args": [],
      "env": {}
    }
  }
}

🎯 Quick Start

CLI Usage

# Health check
vlm health

# Analyze screenshot
vlm describe --image screenshot.png --prompt "What errors are visible?"

# Extract text (OCR)
vlm ocr --image document.png --engine tesseract

# Find UI elements
vlm find --image app.png --query "Submit button"

MCP Usage in Claude

Use vision.describe to analyze this screenshot for errors
Use vision.find_text to locate the "Login" button  
Use vision.ocr to extract all visible text
Use vision.health to check model status

🛠️ Available Tools

`vision.describe`

Analyze images and describe UI state, errors, or issues.

Parameters:

image_b64 (string): Base64-encoded image
prompt (string, optional): Custom analysis prompt
model (string, optional): Ollama model to use
max_tokens (number, optional): Maximum response tokens

Example:

{
  "image_b64": "iVBORw0KGgoAAAANSUhEU...",
  "prompt": "Identify any error messages or broken UI elements",
  "model": "llava:7b"
}

`vision.ocr`

Extract text from images using VLM or Tesseract.

Parameters:

image_b64 (string): Base64-encoded image
engine (string, optional): "vlm" or "tesseract"
model (string, optional): Model for VLM OCR
structured (boolean, optional): Return structured JSON

`vision.find_text`

Locate specific text and return bounding boxes.

Parameters:

image_b64 (string): Base64-encoded image
query (string): Text to search for
model (string, optional): Ollama model to use
fuzzy (boolean, optional): Allow fuzzy matching

`vision.health`

Check Ollama connection and available models.

🔗 Integration Examples

Electron CDP Integration

// Add to your Electron MCP server
server.registerTool(
  "browser_vision_check",
  {
    title: "Analyze page with AI vision",
    inputSchema: z.object({
      prompt: z.string().default("Check for errors"),
      fullPage: z.boolean().default(false)
    })
  },
  async ({ prompt, fullPage }) => {
    const page = await pickPage();
    const screenshot = await page.screenshot({ 
      type: "png", 
      encoding: "base64", 
      fullPage 
    });
    
    // Call Vision MCP via Claude
    return {
      content: [{
        type: "text",
        text: JSON.stringify({
          next_tool: "vision.describe",
          args: { image_b64: screenshot, prompt }
        })
      }]
    };
  }
);

Selenium Integration

// Shell out to vlm CLI from Selenium MCP
async function analyzeSeleniumPage(driver, prompt) {
  const screenshot = await driver.takeScreenshot();
  const tmpPath = `/tmp/selenium-${Date.now()}.png`;
  
  await fs.writeFile(tmpPath, Buffer.from(screenshot, "base64"));
  
  const { execFile } = require("child_process");
  return new Promise((resolve, reject) => {
    execFile("/home/ice/vision-mcp/vlm.mjs",
      ["describe", "--image", tmpPath, "--prompt", prompt],
      (error, stdout) => {
        fs.unlink(tmpPath); // Cleanup
        if (error) reject(error);
        else resolve(JSON.parse(stdout));
      }
    );
  });
}

Playwright in Docker

# docker-compose.yml
services:
  playwright:
    image: mcr.microsoft.com/playwright:v1.54.2-noble
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - OLLAMA_HOST=host.docker.internal

// Direct Ollama call from container
async function analyzeWithVision(page, prompt) {
  const screenshot = await page.screenshot({ encoding: "base64" });
  
  const response = await fetch("http://host.docker.internal:11434/api/generate", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: "llava:7b",
      prompt: `Analyze this UI screenshot: ${prompt}`,
      images: [screenshot],
      stream: false
    })
  });
  
  const data = await response.json();
  return data.response;
}

🤖 Supported Models

Recommended Models by Use Case

Use Case	Model	Size	Strengths
General UI Analysis	`llava:7b` ✅	4.1GB	Reliable, fast, good reasoning
OCR & Text Extraction	`minicpm-v:8b-2.6`	~8GB	State-of-the-art OCR accuracy
Document Analysis	`qwen2.5vl:7b`	~8GB	Excellent for complex layouts
Lightweight	`llava:7b`	4.1GB	Best speed/accuracy balance

Model Installation

# Current default (installed)
ollama pull llava:7b

# Better OCR model
ollama pull minicpm-v:8b-2.6

# Best document analysis (when available)
ollama pull qwen2.5vl:7b

⚙️ Configuration

Environment Variables

Variable	Default	Description
`OLLAMA_HOST`	`127.0.0.1`	Ollama server host
`OLLAMA_PORT`	`11434`	Ollama server port
`VISION_MODEL`	`llava:7b`	Default vision model
`OCR_MODEL`	`llava:7b`	Default OCR model
`MAX_TOKENS`	`1024`	Maximum response tokens

Model Switching

# Use different model temporarily
VISION_MODEL=minicpm-v:8b-2.6 vlm describe --image screenshot.png

# Or specify in tool call
vlm describe --image screenshot.png --model minicpm-v:8b-2.6

🔧 Development

Project Structure

vision-mcp/
├── vision-mcp.mjs          # Main MCP server
├── vlm.mjs                 # CLI tool
├── package.json            # Dependencies
├── setup.sh               # Installation script
├── vision-mcp-wrapper.sh  # MCP wrapper
├── integration-examples.md # Integration guides
└── docs/                  # Additional documentation

Running Development Server

# Start Ollama
ollama serve &

# Test MCP server
node vision-mcp.mjs

# Test CLI tool
./vlm.mjs health

Adding New Models

Pull model: ollama pull model-name
Update defaults in vision-mcp.mjs and vlm.mjs
Test with: vlm describe --model model-name --image test.png

🐛 Troubleshooting

Common Issues

🔴 "Ollama not responding"

ollama serve &
curl http://localhost:11434/api/tags

🔴 "Model not found"

ollama list
ollama pull llava:7b

🔴 "Tesseract not found"

# Fedora/RHEL
sudo dnf install tesseract tesseract-langpack-eng

# Ubuntu/Debian  
sudo apt install tesseract-ocr

🔴 "Permission denied"

chmod +x vision-mcp.mjs vlm.mjs vision-mcp-wrapper.sh

Debug Mode

# Enable debug output
DEBUG=1 ./vlm.mjs describe --image test.png

# Check MCP server logs
journalctl --user -f | grep vision-mcp

Performance Optimization

# Keep models warm (optional)
curl http://localhost:11434/api/generate \
  -d '{"model":"llava:7b","prompt":"warmup","keep_alive":"10m"}'

# Monitor GPU usage
nvidia-smi -l 1

📊 Benchmarks

Performance on 16GB VRAM

Model	Memory Usage	Speed	Accuracy	Best For
`llava:7b`	~4GB	⚡⚡⚡ Fast	⭐⭐⭐ Good	General use, UI analysis
`minicpm-v:8b-2.6`	~8GB	⚡⚡ Medium	⭐⭐⭐⭐ Excellent	OCR, text extraction
`qwen2.5vl:7b`	~8GB	⚡⚡ Medium	⭐⭐⭐⭐ Excellent	Document analysis

🤝 Contributing

Fork the repository
Create a feature branch: git checkout -b feature-name
Commit changes: git commit -am 'Add feature'
Push to branch: git push origin feature-name
Submit a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Ollama - Local model runtime
LLaVA - Vision language model
MCP SDK - Protocol implementation
Claude Code - Development environment

Electron CDP MCP - Electron automation
Selenium MCP - Browser testing
Playwright MCP - Web automation

Made with ❤️ for the Claude Code ecosystem

🌟 Star this repo if you find it helpful! 🐛 Report issues on GitHub 💬 Join discussions in the community

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityInactive

ResponsivenessNo issues

Resources

Need Help?

Related Servers

Related MCP Connectors

Frenchie A
io.github.Lab94
OCR, transcription, file extraction, and image generation for AI agents via MCP.
AI & Machine Learning Image & Video Processing Speech Processing
72
Writingmate MCPOAuth
ai.writingmate
Use AI models for chat, image, and video generation from Claude Code and other MCP hosts.
AI & Machine Learning Image & Video Processing
1
Great ArrowOAuth
ai.greatarrow
Shared memory and actions for Claude, Kiro, OpenAI, Cursor, and other MCP-compatible AI clients.
Knowledge & Memory
1
hithereiamaliff-mcp-nextcloudOAuth
ai.smithery
A comprehensive Model Context Protocol (MCP) server that enables AI assistants to interact with yo…
Cloud Storage File Systems Workplace & Productivity
238

Related MCP Servers

PeepIt MCP
Image & Video Processing OS Automation Autonomous Agents
MantisWare
A
license
A
quality
D
maintenance
Enables AI agents to capture and analyze screenshots of macOS applications, windows, or the entire screen using local (Ollama) or cloud-based AI vision models, with non-intrusive, fast screen capture via Apple's ScreenCaptureKit.
Updated a year ago (2025-06-26 15:17 UTC)
3
5 npm
2
MIT
npu-vision-fallback
Image & Video Processing OS Automation
Byte-Naut
A
license
A
quality
F
maintenance
Provides an MCP server for local low-power screen vision, enabling AI agents to perform OCR and UI detection on inaccessible screens (games, remote desktops) using NPU acceleration and system OCR.
Updated 2 months ago (2026-07-31 16:19 UTC)
5
1
MIT
mcp-vision
Image & Video Processing AI & Machine Learning
MikeyBeez
F
license
A
quality
D
maintenance
MCP server for vision capabilities, enabling screenshot, camera, and image analysis using Ollama vision models.
Updated 8 months ago (2026-01-14 19:30 UTC)
4
1
-
OpenSight MCP
Image & Video Processing AI & Machine Learning
Mr-JoE1
A
license
Not graded
quality
B
maintenance
Multi-backend AI vision for MCP agents. Analyze images, screenshots, and documents using local Ollama models or cloud APIs like OpenAI, Google Gemini, and OpenRouter.
Updated 2 months ago (2026-07-29 11:07 UTC)
MIT

Vision MCP Server 🔍

🚀 Features

📦 Installation

Prerequisites

Quick Setup

Claude Code Integration

🎯 Quick Start

CLI Usage

MCP Usage in Claude

🛠️ Available Tools

vision.describe

vision.ocr

vision.find_text

vision.health

🔗 Integration Examples

Electron CDP Integration

Selenium Integration

Playwright in Docker

🤖 Supported Models

Recommended Models by Use Case

Model Installation

⚙️ Configuration

Environment Variables

Model Switching

🔧 Development

Project Structure

Running Development Server

Adding New Models

🐛 Troubleshooting

Common Issues

Debug Mode

Performance Optimization

📊 Benchmarks

Performance on 16GB VRAM

🤝 Contributing

📄 License

🙏 Acknowledgments

🔗 Related Projects

Maintenance

Resources

Related MCP Connectors

Related MCP Servers

PeepIt MCP

npu-vision-fallback

mcp-vision

OpenSight MCP

`vision.describe`

`vision.ocr`

`vision.find_text`

`vision.health`