PiKVM MCP Server

Give AI agents hands. This MCP server connects Claude Code (or any MCP client) directly to a PiKVM device, giving AI full keyboard, mouse, and screen access to a physical machine -- no browser automation, no virtual desktops, no emulators.

Point it at real hardware. Let the AI see the screen, type commands, click buttons, and navigate GUIs on a machine it could never otherwise touch.

Automatic Mouse Calibration

IP-KVM devices translate mouse coordinates through multiple layers — USB HID emulation, host-side input drivers, display scaling — each introducing positional error. Existing KVM products either ignore this (requiring manual correction) or offer limited auto-sync that only detects cursor acceleration in a fixed corner region.

This MCP server takes a different approach. The pikvm_auto_calibrate tool uses a vision-based algorithm that:

1. Moves the cursor a known distance across multiple randomized screen positions

2. Diffs screenshot pairs to isolate the cursor via connected-component analysis

3. Computes correction factors from detected vs commanded movement using median aggregation

4. Self-verifies by moving to target positions and confirming the cursor lands within 20px

The entire process runs in ~30-60 seconds with no human intervention. Noisy screens (tooltips, animations, dynamic content) are handled through multi-round sampling, ratio divergence filtering, and outlier-resistant statistics — the algorithm discards bad data and still converges on accurate factors.

This is the first IP-KVM tooling — commercial or open source — to implement fully automated mouse coordinate calibration via computer vision. It is what makes precise AI-driven mouse control over a network KVM practical.

See it in action

The video below shows Claude Code using this MCP server to autonomously interact with a Raspberry Pi desktop: taking a screenshot to identify the OS, opening a text editor from the menu, typing text, and closing the application -- all through the PiKVM hardware interface.

This next demonstration shows Claude, connected via the PiKVM MCP server, responding to a natural language prompt to auto-calibrate its mouse coordinate scaling before performing a series of precision mouse tasks on a remote machine. The session concludes with Claude autonomously drawing a house in MS Paint — a simple but effective showcase of accurate, AI-driven input control over an isolated system.

Features

• Automatic mouse calibration — Vision-based cursor detection computes coordinate correction factors with no manual measurement. The first fully automated calibration for IP-KVM.

• Screenshot capture — Get current screen as JPEG image

• Text typing — Type text with proper special character handling via keymaps

• Keyboard control — Send individual keys or key combinations (e.g., Ctrl+Alt+Delete)

• Mouse control — Move, click, and scroll with calibrated coordinate correction

Installation

npm install
npm run build

Configuration

Copy .env.example to .env and configure:

cp .env.example .env

Edit .env:

PIKVM_HOST=https://<your-pikvm-ip>
PIKVM_USERNAME=admin
PIKVM_PASSWORD=your_password
PIKVM_VERIFY_SSL=false
PIKVM_DEFAULT_KEYMAP=en-us

Usage with Claude Code

Requires Node.js 18+. This server uses ES modules. If node --version shows an older version, replace "command": "node" with the full path to a compatible binary (e.g. "/usr/local/bin/node" or your nvm path like "~/.nvm/versions/node/v22.x.x/bin/node"). This is common when nvm's default alias points to an older version.

Add to your Claude Code MCP settings (~/.config/claude-code/settings.json or via the settings UI):

{
  "mcpServers": {
    "pikvm": {
      "command": "node",
      "args": ["/path/to/pikvm_mcp_server/dist/index.js"],
      "env": {
        "PIKVM_HOST": "https://<your-pikvm-ip>",
        "PIKVM_USERNAME": "admin",
        "PIKVM_PASSWORD": "your_password"
      }
    }
  }
}

Or if using the .env file:

{
  "mcpServers": {
    "pikvm": {
      "command": "node",
      "args": ["/path/to/pikvm_mcp_server/dist/index.js"]
    }
  }
}

Available Tools

Display

• pikvm_screenshot - Capture current screen as JPEG (optional: maxWidth, maxHeight, quality)

• pikvm_get_resolution - Get screen resolution and valid coordinate ranges

Keyboard

• pikvm_type - Type text with keymap-aware special character handling (required: text; optional: keymap, slow, delay)

• pikvm_key - Send a key or key combo, e.g. Ctrl+Alt+Del (required: key; optional: modifiers, state)

• pikvm_shortcut - Send multiple keys pressed simultaneously (required: keys array)

Mouse

• pikvm_mouse_move - Move cursor to absolute pixel position or relative delta (required: x, y; optional: relative)

• pikvm_mouse_click - Click a mouse button, optionally at a position (optional: button, x, y, state)

• pikvm_mouse_scroll - Scroll the mouse wheel (required: deltaY; optional: deltaX)

Calibration

• pikvm_auto_calibrate - Automatically detect cursor and compute calibration factors (preferred)

• pikvm_calibrate - Start manual calibration by moving cursor to screen center for visual verification

• pikvm_set_calibration - Apply correction factors calculated from calibration (required: factorX, factorY)

• pikvm_get_calibration - Get current calibration state

• pikvm_clear_calibration - Reset to uncalibrated mode

Skills (Prompts & Skill Tools)

The server exposes 15 skills that provide structured guidance for agents. Each skill is available via two discovery paths:

• MCP Prompts — prompts/list / prompts/get for clients that support the Prompts capability.

• Skill Tools — tools/list / tools/call as skill_* read-only tools, ensuring visibility in marketplaces (e.g. LobeHub) that index tools only.

Tool Guides

Workflow Recipes

See docs/skills/ for detailed human-readable guides.

Key Codes Reference

Common key codes for pikvm_key and pikvm_shortcut:

• Letters: KeyA, KeyB, ... KeyZ

• Numbers: Digit0, Digit1, ... Digit9

• Function keys: F1, F2, ... F12

• Modifiers: ShiftLeft, ShiftRight, ControlLeft, ControlRight, AltLeft, AltRight, MetaLeft, MetaRight

• Special: Enter, Escape, Backspace, Tab, Space, Delete, Insert, Home, End, PageUp, PageDown

• Arrows: ArrowUp, ArrowDown, ArrowLeft, ArrowRight

License

GPL-3.0 - See LICENSE for details.