cvision-mcp

Cross-platform Model Context Protocol (MCP) server for zero-interference background window capture and input automation across Windows, Linux (X11), and Linux (Wayland).

Core Philosophy: Zero-Interference Automation

Standard desktop automation agents (such as Anthropic Computer Use, RobotJS, and PyAutoGUI-based tools) suffer from a critical architectural limitation: they hijack the physical mouse cursor and steal active window focus. When an agent runs using those tools, your cursor jerks across the screen, your active typing session is interrupted, and full-screen video or gaming is broken.

cvision-mcp is engineered from the ground up for silent, background execution:

• No Cursor Drift: Physical hardware pointer coordinates never change.

• No Focus Stealing: Target windows receive synthetic events without changing the active foreground window (GetForegroundWindow / active Wayland seat).

• Background Capture: Screenshots are captured from offscreen or occluded buffers without forcing windows to the front.

• Full Multitasking: Watch 4K video, type, or play games on your primary desktop while the agent works in the background.

Related MCP server: lowlevel-computer-use-mcp

Operating System Architecture

1. Windows (backend/windows.py)

• Implemented entirely in standard Python ctypes (user32.dll, gdi32.dll, kernel32.dll) with zero external C compilation.

• Dispatches mouse and keyboard events directly into the target window's thread message queue via PostMessageW.

• Captures occluded or minimized windows directly to a memory device context (DIBSection) using PrintWindow.

2. Linux X11 & Xwayland (backend/linux_x11.py)

• Employs python-xlib to construct synthetic protocol events (ButtonPress, ButtonRelease, KeyPress, KeyRelease).

• Directs events specifically to the target Window XID with same_screen=1.

• Verified to produce zero displacement of the X11 root pointer.

3. Linux Wayland (backend/linux_wayland.py)

• Wayland's security model isolates client surfaces from inter-client surveillance and global event injection.

• Integrates with KDE Plasma 6 KWin scripting via D-Bus (org.kde.KWin /Scripting) to query window geometry, PIDs, and internal IDs in under 150ms without authorization prompts.

• Performs background window screenshots using headless spectacle -b -n -o <temp> and crops the target bounding box via PIL.

• Provides launch_isolated_app utilizing gamescope (--headless --expose-wayland) to execute apps in a dedicated nested display where input and capture are fully accessible with 0% risk of host desktop interference.

Exposed MCP Tools

list_windows

Lists all accessible desktop windows with dimensions, coordinates, process names, PIDs, and IDs.

• Parameters:

  • filter_title (string, optional): Substring to filter window titles.

• Returns: Array of WindowInfo objects.

capture_window

Captures an unoccluded screenshot of the specified window and saves it to disk as a PNG image.

• Parameters:

  • window_id (string, required): Target window ID (HWND, XID hex 0x..., internal ID, or title substring).

  • output_path (string, optional): Destination path for PNG file.

• Returns: Image metadata including saved path, width, and height.

window_click

Sends a background mouse click to specific relative coordinates (x, y) inside the target window without moving the host cursor.

• Parameters:

  • window_id (string, required): Target window ID or title.

  • x (integer, required): X coordinate relative to the window.

  • y (integer, required): Y coordinate relative to the window.

  • button (string, optional): Mouse button (left, right, middle). Default: left.

  • clicks (integer, optional): Number of clicks (1 for single, 2 for double). Default: 1.

window_type

Types text directly into the target window's message queue without shifting keyboard focus.

• Parameters:

  • window_id (string, required): Target window ID or title.

  • text (string, required): Text string to type.

window_send_key

Sends key press and release sequences (e.g. Enter, Escape, Backspace, Tab, Arrow keys) with optional modifier keys (Ctrl, Shift, Alt).

• Parameters:

  • window_id (string, required): Target window ID or title.

  • key (string, required): Key name (Enter, Escape, Tab, BackSpace, Space, Left, Right, etc.).

  • modifiers (array of strings, optional): List of modifiers (ctrl, shift, alt).

launch_isolated_app

Launches an application in an isolated nested display session (gamescope headless mode on Linux).

• Parameters:

  • command (string, required): Command line to execute.

  • width (integer, optional): Virtual screen width. Default: 1920.

  • height (integer, optional): Virtual screen height. Default: 1080.

Installation & Setup

Requirements

• Python 3.8+

• Pillow (pip install Pillow)

• On Linux: python-xlib (pip install python-xlib)

git clone https://github.com/sorinqu-org/cvision-mcp.git
cd cvision-mcp
pip install -r requirements.txt

MCP Client Configuration

Google Antigravity / Claude Desktop Configuration

Add the server to your mcp_config.json (or claude_desktop_config.json):

{
  "mcpServers": {
    "cvision-mcp": {
      "command": "python3",
      "args": [
        "/path/to/cvision-mcp/server.py"
      ]
    }
  }
}

Windows Configuration

{
  "mcpServers": {
    "cvision-mcp": {
      "command": "python",
      "args": [
        "C:\\path\\to\\cvision-mcp\\server.py"
      ]
    }
  }
}

Protocol Verification

You can verify stdio JSON-RPC 2.0 communication directly from the terminal:

python3 -c "
import subprocess, json
proc = subprocess.Popen(['python3', 'server.py'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True)
proc.stdin.write(json.dumps({'jsonrpc': '2.0', 'id': 1, 'method': 'initialize'}) + '\n')
proc.stdin.flush()
print(proc.stdout.readline())
"

Expected output:

{"jsonrpc": "2.0", "id": 1, "result": {"protocolVersion": "2024-11-05", "capabilities": {"tools": {}}, "serverInfo": {"name": "cvision-mcp", "version": "1.0.0"}}}

License

Apache-2.0 License. See LICENSE for details.