Skip to main content
Glama
VNC MCP Server
by volkan-m

VNC MCP Server

An advanced Model Context Protocol (MCP) server that empowers AI agents to see, reason, and control computers via the VNC (RFB) protocol. This server transforms a standard VNC connection into a high-level "Vision-Action" interface for AI Agents.

📊 System Architecture

graph TD
    subgraph "AI Environment"
        A[AI Agent / Claude Desktop] -->|MCP Protocol| B[VNC MCP Server]
    end

    subgraph "Bridge Layer"
        B -->|Image Processing| C[Sharp / Tesseract.js]
        B -->|RFB Protocol| D[VNC Connection]
    end

    subgraph "Target System"
        D -->|Input Simulation| E[Windows / macOS / Linux]
        E -->|Framebuffer| D
    end

    style B fill:#f00,stroke:#333,stroke-width:4px

Related MCP server: servo-mcp

🌟 Key Features

đŸ‘ī¸ Advanced Vision

  • Regional Screenshot: Precise screen capture of specific areas to save tokens.

  • OCR (Optical Character Recognition): Extract text from any part of the screen using Tesseract.js.

  • Visual Search: Find specific icons, buttons, or images on the screen using template matching.

  • Pixel Analysis: Get exact Hex/RGB colors of any pixel for state verification.

  • Change Detection: Wait for a specific region to update before proceeding.

đŸ•šī¸ Human-Like Control

  • Precise Input: Single clicks, double clicks, and smooth mouse movement.

  • Drag & Drop: Simulate complex dragging operations between coordinates.

  • Scrolling: Directional mouse wheel support (Up/Down/Left/Right).

  • Keyboard Mastery: Type strings and perform complex key combinations (e.g., Ctrl+C, Cmd+Space).

⚡ Hybrid Control (VNC + SSH Shell)

  • Background Execution: Run any shell command (bash/cmd/powershell) on the server.

  • 🔗 Remote Execution via SSH: Run commands directly on the VNC host via integrated SSH client.

  • đŸ›Ąī¸ Security Layer: Robust filtering system with platform-specific allowlists and denylists.

  • Cross-Platform Support: Works on Windows, Linux, and macOS.

  • Seamless Integration: Use terminal to install software then use VNC to control apps.

🧠 Intelligence & Health

  • OS Detection & Platform Awareness: Intelligent OS detection based on screen resolution patterns (Windows/macOS/Linux).

  • Connection Health: Real-time connectivity monitoring with latency measurement and heartbeat status.

  • Clipboard Sync: Full bidirectional clipboard support.

  • Auto-Connect: Configure server to connect immediately via environment variables.

  • Advanced Sequencing: Execute multiple actions in a single call for faster automation.

🚀 Getting Started

Prerequisites

  • Node.js v18+

  • A running VNC server on the target system.

  • (Optional) SSH service enabled on the target system for remote command execution.

Quick Start

  1. Clone & Install:

    git clone https://github.com/volkan-m/vnc-mcp-server.git
    cd vnc-mcp-server
    npm install
    npm run build
  2. Test Environment (Docker with Chrome & SSH): Run a full Linux desktop with Google Chrome, VNC, and SSH enabled:

    docker run -d \
      -p 5901:5900 \
      -p 2222:22 \
      -v /dev/shm:/dev/shm \
      --name mcp-vnc-chrome \
      -e VNC_PASSWORD=vncpass \
      dorowu/ubuntu-desktop-lxde-vnc:latest

    Connect to VNC at localhost:5901 and SSH at localhost:2222 (port 22 inside).

  3. MCP Client Configuration:

Google Antigravity / Claude Desktop

Add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "vnc": {
      "command": "node",
      "args": ["/absolute/path/to/vnc-mcp-server/dist/index.js"],
      "env": {
        "VNC_HOST": "localhost",
        "VNC_PORT": "5901",
        "VNC_PASSWORD": "vncpass"
      }
    }
  }
}

đŸ›Ąī¸ Security Configuration (Optional)

You can restrict which system commands the AI can execute:

Variable

Description

Default Denied

LINUX_DENIED_COMMANDS

Blocked Linux commands

`rm -rf,rm -f,shred,mkfs,:(){ :

WIN_DENIED_COMMANDS

Blocked Windows commands

del,format,rd,sfc,attrib

MAC_DENIED_COMMANDS

Blocked macOS commands

rm -rf,rm -f

[OS]_ALLOWED_COMMANDS

If set, ONLY these are allowed

(Empty by default)

🔗 Remote Execution via SSH

The server now supports direct command execution on the target machine via SSH. When connecting through vnc_connect, you can provide an optional ssh object:

{
  "host": "192.168.1.50",
  "ssh": {
    "user": "root",
    "password": "your-password",
    "port": 22
  }
}

If SSH is connected, vnc_system_command automatically routes all commands to the target machine instead of the local server.

📊 Enhanced Status Monitoring

Screen Info

vnc_get_screen_info returns detailed information about the remote session:

  • Resolution: Current display dimensions

  • OS Hint: Intelligent OS detection (Windows/macOS/Linux based on resolution patterns)

  • SSH Status: Whether SSH connection is active

Connection Health

vnc_get_connection_health provides comprehensive connectivity metrics:

  • VNC Status: Connected/Disconnected state

  • Latency: Real-time latency measurement from screen refresh operations

  • Resolution: Current display dimensions

  • SSH Status: SSH connection state

đŸŽ¯ Advanced Vision Features

Screenshot with Coordinate Overlay

Use the overlay parameter in vnc_screenshot to add a grid with labeled coordinates (A1, B2, etc.) for easier navigation:

{
  "format": "jpeg",
  "quality": 85,
  "overlay": true
}

This helps AI agents understand and communicate screen positions more accurately.

OCR with Multiple Languages

The vnc_ocr_region tool supports multiple languages via Tesseract.js. Specify the language code:

{
  "x": 0,
  "y": 0,
  "width": 1920,
  "height": 1080,
  "lang": "eng+deu+fra"
}

Supported languages: eng (English), deu (German), fra (French), spa (Spanish), and many more.

Batch Actions with vnc_execute_sequence

Execute multiple input actions in a single call for faster automation:

{
  "actions": [
    {"type": "move", "x": 100, "y": 100},
    {"type": "click", "x": 100, "y": 100},
    {"type": "wait", "ms": 500},
    {"type": "type", "text": "Hello World"},
    {"type": "key", "key": "Enter"}
  ]
}

This approach reduces RPC overhead and improves responsiveness for complex automation sequences.

🧰 API Reference (Tools)

Tool

Category

Description

vnc_connect

Setup

Connect to VNC (optional: include ssh configuration).

vnc_disconnect

Setup

Disconnect from VNC server.

vnc_get_screen_info

Status

Returns resolution, OS hint (Windows/macOS/Linux), and SSH status.

vnc_get_connection_health

Status

Provides heartbeat status, latency measurement, resolution, and SSH connection state.

vnc_screenshot

Vision

Captures PNG/JPEG of full screen or region with optional coordinate overlay.

vnc_ocr_region

Vision

Extracts text from a specific region with OCR (supports multiple languages).

vnc_find_image

Vision

Finds a template image on screen with configurable tolerance.

vnc_wait_for_change

Vision

Blocks until a screen region updates within timeout.

vnc_get_pixel_color

Vision

Returns Hex/RGB color value of a specific pixel.

vnc_mouse_move

Input

Moves mouse to coordinates.

vnc_mouse_click

Input

Performs mouse clicks (Left/Middle/Right buttons).

vnc_mouse_double_click

Input

Performs a quick double click.

vnc_mouse_drag

Input

Drags from point A to point B with button control.

vnc_mouse_scroll

Input

Directional wheel scrolling (Up/Down/Left/Right).

vnc_key_tap

Input

Taps keys with modifiers (e.g., C-v, M-Tab, S-Enter).

vnc_type_string

Input

Types a sequence of characters.

vnc_execute_sequence

Input

Executes multiple input actions in a single call for performance.

vnc_system_command

System

Execute shell commands (Local or Remote via SSH).

vnc_launch_gui_app

System

Launches GUI applications on the remote desktop.

vnc_set_clipboard

System

Sets remote system clipboard text.

vnc_get_clipboard

System

Retrieves last known remote clipboard content.

🛡 License

MIT License. See LICENSE for details.

Available Tools

21 tools
vnc_connectC

Connect to a VNC server

Inputs: host (required), ssh, port, password.

vnc_disconnectA

Disconnect from the VNC server

Inputs: no named parameters.

vnc_execute_sequenceA

Execute a sequence of input actions (move, click, type, key, wait) in a single call for higher speed

Inputs: actions (required).

vnc_find_imageC

Find a template image on the screen

Inputs: template_base64 (required), region, tolerance.

vnc_get_clipboardA

Get the last captured clipboard text

Inputs: no named parameters.

vnc_get_connection_healthB

Get connection health metrics

Inputs: no named parameters.

vnc_get_pixel_colorD

Get the color of a pixel

Inputs: x (required), y (required).

vnc_get_screen_infoA

Get screen resolution and OS hint

Inputs: no named parameters.

vnc_key_tapA

Tap a key (e.g., 'Enter', 'C-c')

Inputs: key (required).

vnc_launch_gui_appB

Launch a GUI application on the remote desktop (Linux focus)

Inputs: app_command (required), args.

vnc_mouse_clickD

Click the mouse

Inputs: x (required), y (required), button.

vnc_mouse_double_clickC

Double click the mouse

Inputs: x (required), y (required), button.

vnc_mouse_dragC

Perform a drag and drop

Inputs: endX (required), endY (required), startX (required), startY (required), +1 more.

vnc_mouse_moveD

Move the mouse

Inputs: x (required), y (required).

vnc_mouse_scrollC

Scroll the mouse

Inputs: direction (required), amount.

vnc_ocr_regionB

Extract text from a specific region using OCR

Inputs: x (required), y (required), width (required), height (required), +1 more.

vnc_screenshotC

Capture a screenshot

Inputs: format, region, overlay, quality.

vnc_set_clipboardC

Set the remote clipboard

Inputs: text (required).

vnc_system_commandB

Execute a shell command in the background on the server

Inputs: command (required).

vnc_type_stringC

Type a string

Inputs: text (required).

vnc_wait_for_changeC

Wait until a region changes

Inputs: x (required), y (required), width (required), height (required), +1 more.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelogâ€Ļ

    TDQS

    B3/5.0

    Scored across 21 tools

    Disambiguation5/5

    Each tool targets a distinct action on the VNC server, such as connection management, mouse/keyboard input, screen analysis, clipboard, and system commands. The boundaries between tools are clear, with no overlapping purposes that would cause misselection.

    Naming Consistency5/5

    All tools follow a consistent 'vnc_' prefix with snake_case verb_noun naming (e.g., vnc_connect, vnc_mouse_move, vnc_get_screen_info). The pattern is uniform and predictable, with getters consistently using 'get_'.

    Tool Count4/5

    At 21 tools, the count is slightly above the typical well-scoped range (3-15), but each tool serves a distinct and necessary function for remote desktop automation. The slight heaviness is justified by the breadth of VNC operations covered.

    Completeness5/5

    The tool set covers the full spectrum of VNC operations: connect/disconnect, screen capture and analysis (screenshot, OCR, image finding), input methods (mouse, keyboard, typing), clipboard access, system commands, and GUI launching. There are no obvious missing operations that would hinder normal workflows.

    Maintenance

    ActivityInactive
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      D
      maintenance
      Enables MCP-compatible LLMs to interact with any desktop accessible over VNC, providing tools for screen reading (OCR), mouse and keyboard control, and automation.
      13
      AGPL 3.0
    • F
      license
      Not graded
      quality
      D
      maintenance
      Enables AI agents to see and control your desktop with tools for screenshots, clicks, typing, and more, all locally on macOS and Windows.
      1
      -
    • A
      license
      A
      quality
      D
      maintenance
      Enables AI agents to remotely control computers via VNC, including mouse movement, clicking, keyboard input, text entry, and screen capture across Windows, Linux, and macOS systems.
      6
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to control remote desktops through screen capture, mouse movement, and keyboard input.
      MIT