mcp-serial-hid-kvm

MCP (Model Context Protocol) server that gives AI agents full keyboard, mouse, and screen access to a physical PC. Thin client for serial-hid-kvm — all hardware control is delegated via TCP.

How It Works

Claude / AI Agent
  ↕ MCP (stdio)
mcp-serial-hid-kvm        ← this package (thin client + OCR)
  ↕ TCP (localhost:9329)
serial-hid-kvm             ← standalone KVM server (owns hardware)
  ↕ USB Serial + HDMI
Target PC

The KVM server (serial-hid-kvm) runs as a persistent process owning the serial port and capture device. This MCP server connects to it as a TCP client. Multiple MCP instances (multiple Claude sessions) can share a single KVM server without device conflicts.

Related MCP server: AI Gaming Agent MCP Server

Prerequisites

1. Hardware: CH9329+CH340 USB HID cable + USB HDMI capture device (see serial-hid-kvm for details)

2. serial-hid-kvm installed and running:

   pip install -e /path/to/serial-hid-kvm
   serial-hid-kvm --api              # with preview window
   serial-hid-kvm --api --headless   # or headless

3. Tesseract OCR (for get_screen_text / execute_and_read):

   • Linux: sudo apt install tesseract-ocr

   • Windows: https://github.com/tesseract-ocr/tesseract

Installation

pip install -e .

This automatically installs serial-hid-kvm as a dependency.

MCP Client Configuration

Claude Desktop / Claude Code

{
  "mcpServers": {
    "kvm": {
      "command": "mcp-serial-hid-kvm"
    }
  }
}

Custom KVM server address:

{
  "mcpServers": {
    "kvm": {
      "command": "mcp-serial-hid-kvm",
      "env": {
        "SHKVM_API_HOST": "127.0.0.1",
        "SHKVM_API_PORT": "9329"
      }
    }
  }
}

Configuration

Hardware settings (SHKVM_SERIAL_PORT, SHKVM_SCREEN_WIDTH, etc.) are configured on the KVM server side, not here. If the target PC uses a non-US keyboard, set --target-layout (or SHKVM_TARGET_LAYOUT) on the KVM server so that type_text and send_key produce correct characters.

Available Tools

Keyboard

Mouse

Screen

Device Management

Architecture

This package is intentionally minimal (~4 files):

mcp_serial_hid_kvm/
  server.py    MCP tool handlers → KvmClient TCP calls
  config.py    KVM host/port, tesseract, log settings
  ocr.py       Tesseract OCR (runs locally on fetched frames)
  __init__.py

All keyboard/mouse/capture logic lives in serial-hid-kvm. This package only translates MCP tool calls to TCP API calls and runs OCR locally.

Why Separate?

• No device conflicts — multiple Claude sessions share one KVM server

• Independent restarts — restart the MCP server without losing the KVM connection

• Standalone use — serial-hid-kvm works without MCP (interactive preview, scripts, other AI frameworks)

License

MIT