# cvision-mcp

Cross-platform Model Context Protocol (MCP) server for zero-interference background window capture and input automation across Windows, Linux (X11), and Linux (Wayland).

## Core Philosophy: Zero-Interference Automation

Standard desktop automation agents (such as Anthropic Computer Use, RobotJS, and PyAutoGUI-based tools) suffer from a critical architectural limitation: **they hijack the physical mouse cursor and steal active window focus**. When an agent runs using those tools, your cursor jerks across the screen, your active typing session is interrupted, and full-screen video or gaming is broken.

`cvision-mcp` is engineered from the ground up for **silent, background execution**:

- **No Cursor Drift**: Physical hardware pointer coordinates never change.
- **No Focus Stealing**: Target windows receive synthetic events without changing the active foreground window (`GetForegroundWindow` / active Wayland seat).
- **Background Capture**: Screenshots are captured from offscreen or occluded buffers without forcing windows to the front.
- **Full Multitasking**: Watch 4K video, type, or play games on your primary desktop while the agent works in the background.

---

## Operating System Architecture

| Feature | Windows | Linux (X11 / Xwayland) | Linux (Wayland / KDE) |
| :--- | :--- | :--- | :--- |
| **Window Enumeration** | `EnumWindows` + `GetWindowRect` | `_NET_CLIENT_LIST` via Xlib | KWin D-Bus Scripting (`qdbus6`) |
| **Screenshot Capture** | `PrintWindow(PW_RENDERFULLCONTENT)` | `spectacle -b -n` / `XGetImage` | Headless Spectacle + Bounding Box Crop |
| **Mouse Click Injection** | `PostMessageW(WM_LBUTTONDOWN / UP)` | `XSendEvent(ButtonPress / Release)` | XSendEvent (Xwayland) / Gamescope |
| **Keyboard Injection** | `PostMessageW(WM_CHAR / WM_KEYDOWN)`| `XSendEvent(KeyPress / KeyRelease)` | XSendEvent (Xwayland) / Gamescope |
| **Isolated Session Mode**| Dedicated Desktop Object | Virtual Display (`Xvfb`) | Headless `gamescope` Nested Compositor |

### 1. Windows (`backend/windows.py`)
- Implemented entirely in standard Python `ctypes` (`user32.dll`, `gdi32.dll`, `kernel32.dll`) with zero external C compilation.
- Dispatches mouse and keyboard events directly into the target window's thread message queue via `PostMessageW`.
- Captures occluded or minimized windows directly to a memory device context (DIBSection) using `PrintWindow`.

### 2. Linux X11 & Xwayland (`backend/linux_x11.py`)
- Employs `python-xlib` to construct synthetic protocol events (`ButtonPress`, `ButtonRelease`, `KeyPress`, `KeyRelease`).
- Directs events specifically to the target `Window` XID with `same_screen=1`.
- Verified to produce zero displacement of the X11 root pointer.

### 3. Linux Wayland (`backend/linux_wayland.py`)
- Wayland's security model isolates client surfaces from inter-client surveillance and global event injection.
- Integrates with KDE Plasma 6 KWin scripting via D-Bus (`org.kde.KWin /Scripting`) to query window geometry, PIDs, and internal IDs in under 150ms without authorization prompts.
- Performs background window screenshots using headless `spectacle -b -n -o <temp>` and crops the target bounding box via PIL.
- Provides `launch_isolated_app` utilizing `gamescope` (`--headless --expose-wayland`) to execute apps in a dedicated nested display where input and capture are fully accessible with 0% risk of host desktop interference.

---

## Exposed MCP Tools

### `list_windows`
Lists all accessible desktop windows with dimensions, coordinates, process names, PIDs, and IDs.
- **Parameters**:
  - `filter_title` *(string, optional)*: Substring to filter window titles.
- **Returns**: Array of `WindowInfo` objects.

### `capture_window`
Captures an unoccluded screenshot of the specified window and saves it to disk as a PNG image.
- **Parameters**:
  - `window_id` *(string, required)*: Target window ID (HWND, XID hex `0x...`, internal ID, or title substring).
  - `output_path` *(string, optional)*: Destination path for PNG file.
- **Returns**: Image metadata including saved path, width, and height.

### `window_click`
Sends a background mouse click to specific relative coordinates `(x, y)` inside the target window without moving the host cursor.
- **Parameters**:
  - `window_id` *(string, required)*: Target window ID or title.
  - `x` *(integer, required)*: X coordinate relative to the window.
  - `y` *(integer, required)*: Y coordinate relative to the window.
  - `button` *(string, optional)*: Mouse button (`left`, `right`, `middle`). Default: `left`.
  - `clicks` *(integer, optional)*: Number of clicks (1 for single, 2 for double). Default: 1.

### `window_type`
Types text directly into the target window's message queue without shifting keyboard focus.
- **Parameters**:
  - `window_id` *(string, required)*: Target window ID or title.
  - `text` *(string, required)*: Text string to type.

### `window_send_key`
Sends key press and release sequences (e.g. Enter, Escape, Backspace, Tab, Arrow keys) with optional modifier keys (Ctrl, Shift, Alt).
- **Parameters**:
  - `window_id` *(string, required)*: Target window ID or title.
  - `key` *(string, required)*: Key name (`Enter`, `Escape`, `Tab`, `BackSpace`, `Space`, `Left`, `Right`, etc.).
  - `modifiers` *(array of strings, optional)*: List of modifiers (`ctrl`, `shift`, `alt`).

### `launch_isolated_app`
Launches an application in an isolated nested display session (`gamescope` headless mode on Linux).
- **Parameters**:
  - `command` *(string, required)*: Command line to execute.
  - `width` *(integer, optional)*: Virtual screen width. Default: 1920.
  - `height` *(integer, optional)*: Virtual screen height. Default: 1080.

---

## Installation & Setup

### Requirements
- Python 3.8+
- Pillow (`pip install Pillow`)
- On Linux: `python-xlib` (`pip install python-xlib`)

```bash
git clone https://github.com/sorinqu-org/cvision-mcp.git
cd cvision-mcp
pip install -r requirements.txt
```

---

## MCP Client Configuration

### Google Antigravity / Claude Desktop Configuration
Add the server to your `mcp_config.json` (or `claude_desktop_config.json`):

```json
{
  "mcpServers": {
    "cvision-mcp": {
      "command": "python3",
      "args": [
        "/path/to/cvision-mcp/server.py"
      ]
    }
  }
}
```

### Windows Configuration
```json
{
  "mcpServers": {
    "cvision-mcp": {
      "command": "python",
      "args": [
        "C:\\path\\to\\cvision-mcp\\server.py"
      ]
    }
  }
}
```

---

## Protocol Verification

You can verify stdio JSON-RPC 2.0 communication directly from the terminal:

```bash
python3 -c "
import subprocess, json
proc = subprocess.Popen(['python3', 'server.py'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True)
proc.stdin.write(json.dumps({'jsonrpc': '2.0', 'id': 1, 'method': 'initialize'}) + '\n')
proc.stdin.flush()
print(proc.stdout.readline())
"
```

Expected output:
```json
{"jsonrpc": "2.0", "id": 1, "result": {"protocolVersion": "2024-11-05", "capabilities": {"tools": {}}, "serverInfo": {"name": "cvision-mcp", "version": "1.0.0"}}}
```

---

## License

Apache-2.0 License. See `LICENSE` for details.