VNC MCP Server
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@VNC MCP Server take a screenshot and OCR the text"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
VNC MCP Server
An advanced Model Context Protocol (MCP) server that empowers AI agents to see, reason, and control computers via the VNC (RFB) protocol. This server transforms a standard VNC connection into a high-level "Vision-Action" interface for AI Agents.
đ System Architecture
graph TD
subgraph "AI Environment"
A[AI Agent / Claude Desktop] -->|MCP Protocol| B[VNC MCP Server]
end
subgraph "Bridge Layer"
B -->|Image Processing| C[Sharp / Tesseract.js]
B -->|RFB Protocol| D[VNC Connection]
end
subgraph "Target System"
D -->|Input Simulation| E[Windows / macOS / Linux]
E -->|Framebuffer| D
end
style B fill:#f00,stroke:#333,stroke-width:4pxRelated MCP server: servo-mcp
đ Key Features
đī¸ Advanced Vision
Regional Screenshot: Precise screen capture of specific areas to save tokens.
OCR (Optical Character Recognition): Extract text from any part of the screen using Tesseract.js.
Visual Search: Find specific icons, buttons, or images on the screen using template matching.
Pixel Analysis: Get exact Hex/RGB colors of any pixel for state verification.
Change Detection: Wait for a specific region to update before proceeding.
đšī¸ Human-Like Control
Precise Input: Single clicks, double clicks, and smooth mouse movement.
Drag & Drop: Simulate complex dragging operations between coordinates.
Scrolling: Directional mouse wheel support (Up/Down/Left/Right).
Keyboard Mastery: Type strings and perform complex key combinations (e.g.,
Ctrl+C,Cmd+Space).
⥠Hybrid Control (VNC + SSH Shell)
Background Execution: Run any shell command (bash/cmd/powershell) on the server.
đ Remote Execution via SSH: Run commands directly on the VNC host via integrated SSH client.
đĄī¸ Security Layer: Robust filtering system with platform-specific allowlists and denylists.
Cross-Platform Support: Works on Windows, Linux, and macOS.
Seamless Integration: Use terminal to install software then use VNC to control apps.
đ§ Intelligence & Health
OS Detection & Platform Awareness: Intelligent OS detection based on screen resolution patterns (Windows/macOS/Linux).
Connection Health: Real-time connectivity monitoring with latency measurement and heartbeat status.
Clipboard Sync: Full bidirectional clipboard support.
Auto-Connect: Configure server to connect immediately via environment variables.
Advanced Sequencing: Execute multiple actions in a single call for faster automation.
đ Getting Started
Prerequisites
Node.js v18+
A running VNC server on the target system.
(Optional) SSH service enabled on the target system for remote command execution.
Quick Start
Clone & Install:
git clone https://github.com/volkan-m/vnc-mcp-server.git cd vnc-mcp-server npm install npm run buildTest Environment (Docker with Chrome & SSH): Run a full Linux desktop with Google Chrome, VNC, and SSH enabled:
docker run -d \ -p 5901:5900 \ -p 2222:22 \ -v /dev/shm:/dev/shm \ --name mcp-vnc-chrome \ -e VNC_PASSWORD=vncpass \ dorowu/ubuntu-desktop-lxde-vnc:latestConnect to VNC at localhost:5901 and SSH at localhost:2222 (port 22 inside).
MCP Client Configuration:
Google Antigravity / Claude Desktop
Add this to your claude_desktop_config.json:
{
"mcpServers": {
"vnc": {
"command": "node",
"args": ["/absolute/path/to/vnc-mcp-server/dist/index.js"],
"env": {
"VNC_HOST": "localhost",
"VNC_PORT": "5901",
"VNC_PASSWORD": "vncpass"
}
}
}
}đĄī¸ Security Configuration (Optional)
You can restrict which system commands the AI can execute:
Variable | Description | Default Denied |
| Blocked Linux commands | `rm -rf,rm -f,shred,mkfs,:(){ : |
| Blocked Windows commands |
|
| Blocked macOS commands |
|
| If set, ONLY these are allowed | (Empty by default) |
đ Remote Execution via SSH
The server now supports direct command execution on the target machine via SSH. When connecting through vnc_connect, you can provide an optional ssh object:
{
"host": "192.168.1.50",
"ssh": {
"user": "root",
"password": "your-password",
"port": 22
}
}If SSH is connected, vnc_system_command automatically routes all commands to the target machine instead of the local server.
đ Enhanced Status Monitoring
Screen Info
vnc_get_screen_info returns detailed information about the remote session:
Resolution: Current display dimensions
OS Hint: Intelligent OS detection (Windows/macOS/Linux based on resolution patterns)
SSH Status: Whether SSH connection is active
Connection Health
vnc_get_connection_health provides comprehensive connectivity metrics:
VNC Status: Connected/Disconnected state
Latency: Real-time latency measurement from screen refresh operations
Resolution: Current display dimensions
SSH Status: SSH connection state
đ¯ Advanced Vision Features
Screenshot with Coordinate Overlay
Use the overlay parameter in vnc_screenshot to add a grid with labeled coordinates (A1, B2, etc.) for easier navigation:
{
"format": "jpeg",
"quality": 85,
"overlay": true
}This helps AI agents understand and communicate screen positions more accurately.
OCR with Multiple Languages
The vnc_ocr_region tool supports multiple languages via Tesseract.js. Specify the language code:
{
"x": 0,
"y": 0,
"width": 1920,
"height": 1080,
"lang": "eng+deu+fra"
}Supported languages: eng (English), deu (German), fra (French), spa (Spanish), and many more.
Batch Actions with vnc_execute_sequence
Execute multiple input actions in a single call for faster automation:
{
"actions": [
{"type": "move", "x": 100, "y": 100},
{"type": "click", "x": 100, "y": 100},
{"type": "wait", "ms": 500},
{"type": "type", "text": "Hello World"},
{"type": "key", "key": "Enter"}
]
}This approach reduces RPC overhead and improves responsiveness for complex automation sequences.
đ§° API Reference (Tools)
Tool | Category | Description |
| Setup | Connect to VNC (optional: include |
| Setup | Disconnect from VNC server. |
| Status | Returns resolution, OS hint (Windows/macOS/Linux), and SSH status. |
| Status | Provides heartbeat status, latency measurement, resolution, and SSH connection state. |
| Vision | Captures PNG/JPEG of full screen or region with optional coordinate overlay. |
| Vision | Extracts text from a specific region with OCR (supports multiple languages). |
| Vision | Finds a template image on screen with configurable tolerance. |
| Vision | Blocks until a screen region updates within timeout. |
| Vision | Returns Hex/RGB color value of a specific pixel. |
| Input | Moves mouse to coordinates. |
| Input | Performs mouse clicks (Left/Middle/Right buttons). |
| Input | Performs a quick double click. |
| Input | Drags from point A to point B with button control. |
| Input | Directional wheel scrolling (Up/Down/Left/Right). |
| Input | Taps keys with modifiers (e.g., |
| Input | Types a sequence of characters. |
| Input | Executes multiple input actions in a single call for performance. |
| System | Execute shell commands (Local or Remote via SSH). |
| System | Launches GUI applications on the remote desktop. |
| System | Sets remote system clipboard text. |
| System | Retrieves last known remote clipboard content. |
đĄ License
MIT License. See LICENSE for details.
Available Tools
21 toolsvnc_connectC
Connect to a VNC server
Inputs: host (required), ssh, port, password.
vnc_disconnectA
Disconnect from the VNC server
Inputs: no named parameters.
vnc_execute_sequenceA
Execute a sequence of input actions (move, click, type, key, wait) in a single call for higher speed
Inputs: actions (required).
vnc_find_imageC
Find a template image on the screen
Inputs: template_base64 (required), region, tolerance.
vnc_get_clipboardA
Get the last captured clipboard text
Inputs: no named parameters.
vnc_get_connection_healthB
Get connection health metrics
Inputs: no named parameters.
vnc_get_pixel_colorD
Get the color of a pixel
Inputs: x (required), y (required).
vnc_get_screen_infoA
Get screen resolution and OS hint
Inputs: no named parameters.
vnc_key_tapA
Tap a key (e.g., 'Enter', 'C-c')
Inputs: key (required).
vnc_launch_gui_appB
Launch a GUI application on the remote desktop (Linux focus)
Inputs: app_command (required), args.
vnc_mouse_clickD
Click the mouse
Inputs: x (required), y (required), button.
vnc_mouse_double_clickC
Double click the mouse
Inputs: x (required), y (required), button.
vnc_mouse_dragC
Perform a drag and drop
Inputs: endX (required), endY (required), startX (required), startY (required), +1 more.
vnc_mouse_moveD
Move the mouse
Inputs: x (required), y (required).
vnc_mouse_scrollC
Scroll the mouse
Inputs: direction (required), amount.
vnc_ocr_regionB
Extract text from a specific region using OCR
Inputs: x (required), y (required), width (required), height (required), +1 more.
vnc_screenshotC
Capture a screenshot
Inputs: format, region, overlay, quality.
vnc_set_clipboardC
Set the remote clipboard
Inputs: text (required).
vnc_system_commandB
Execute a shell command in the background on the server
Inputs: command (required).
vnc_type_stringC
Type a string
Inputs: text (required).
vnc_wait_for_changeC
Wait until a region changes
Inputs: x (required), y (required), width (required), height (required), +1 more.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelogâĻ
TDQS
Scored across 21 tools
Each tool targets a distinct action on the VNC server, such as connection management, mouse/keyboard input, screen analysis, clipboard, and system commands. The boundaries between tools are clear, with no overlapping purposes that would cause misselection.
All tools follow a consistent 'vnc_' prefix with snake_case verb_noun naming (e.g., vnc_connect, vnc_mouse_move, vnc_get_screen_info). The pattern is uniform and predictable, with getters consistently using 'get_'.
At 21 tools, the count is slightly above the typical well-scoped range (3-15), but each tool serves a distinct and necessary function for remote desktop automation. The slight heaviness is justified by the breadth of VNC operations covered.
The tool set covers the full spectrum of VNC operations: connect/disconnect, screen capture and analysis (screenshot, OCR, image finding), input methods (mouse, keyboard, typing), clipboard access, system commands, and GUI launching. There are no obvious missing operations that would hinder normal workflows.
Maintenance
Related MCP Connectors
Real desktops for AI agents, on machines you choose: screenshot, type, click, hire more agents.
Connect AI assistants to your approved computer for files, commands, browser and desktop tasks.
Reach your own computers from any AI app: files, commands, screen and mouse, approvals and alerts.
Cloud desktops for AI agents: create a Linux computer, see its screen, click, type and run tasks.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceEnables MCP-compatible LLMs to interact with any desktop accessible over VNC, providing tools for screen reading (OCR), mouse and keyboard control, and automation.13AGPL 3.0
- FlicenseNot gradedqualityDmaintenanceEnables AI agents to see and control your desktop with tools for screenshots, clicks, typing, and more, all locally on macOS and Windows.1-
- AlicenseAqualityDmaintenanceEnables AI agents to remotely control computers via VNC, including mouse movement, clicking, keyboard input, text entry, and screen capture across Windows, Linux, and macOS systems.6MIT
- AlicenseNot gradedqualityBmaintenanceEnables AI agents to control remote desktops through screen capture, mouse movement, and keyboard input.MIT