gamepad-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@gamepad-mcp hold the left stick up for 2 seconds, then press cross"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
gamepad-mcp
A scriptable gamepad for AI agents: an MCP server that plays games by posting real HID keyboard and mouse events on the host machine.
Why this exists
Driving a game from an agent usually dies at the same wall: synthetic input APIs that go through accessibility layers need the target window to expose a keyboard recipient, and game/streaming windows — Chiaki's stream view is the one that taught us this — often don't. A physical keyboard works because its events enter at the HID layer.
gamepad-mcp posts events at that layer (macOS: CGEventPost at the HID
event tap). To any window, its input is indistinguishable from a person
typing. On top of that transport it exposes a DualSense-shaped tool
surface — press("cross"), hold_stick("left", "up", 2000) — translated
through a keymap profile for whatever client is actually receiving the keys
(Chiaki-ng ships built in; any keyboard-driven game works with a profile you
write in five minutes).
Related MCP server: Parallel CUA
Install
./install.sh # venv at ~/.local/share/gamepad-mcp, link in ~/.local/binOn macOS, one permission is required — the server posts HID events, which macOS gates behind Accessibility trust:
Start the server once (below).
System Settings → Privacy & Security → Accessibility → enable the terminal/python entry running gamepad-mcp. (On macOS 27 this list lives under Privacy & Security → Device Control and Data Access instead.) The grant only latches after the server process restarts.
Call the
statustool:"trusted": truemeans you're ready.
Run
# stdio — for an agent on the same machine
gamepad-mcp
# streamable HTTP — for a remote agent; bind a private address (e.g. your
# Tailscale IP), never a public interface, and always set a token
gamepad-mcp --http --host 100.x.y.z --port 8765 --token <secret>
# dry-run anywhere (records events instead of posting them)
gamepad-mcp --backend null
# public mode — when the terminal might be screenshotted or streamed:
# IPv4 addresses in the log output are masked (every octet but the last
# becomes X's, ports stay readable), e.g. uvicorn reports
# http://XXX.XXX.XXX.21:8765. Display-only; the bind is unchanged.
# Also settable as GAMEPAD_PUBLIC=1.
gamepad-mcp --http --host 100.x.y.z --port 8765 --token <secret> --publicAn optional launchd agent lives in launchd/com.gamepadmcp.server.plist.
MCP client config (stdio):
{"mcpServers": {"gamepad": {"command": "gamepad-mcp",
"env": {"GAMEPAD_PROFILE": "chiaki-ng"}}}}Tools
Tool | What it does |
| Backend, Accessibility trust, active profile, currently held keys |
| DualSense buttons: |
|
|
| Raw keys by name ( |
| Chords, e.g. |
| Panic button: every held key up, now |
| Absolute pointer control |
| Keymap management |
Safety model
Every key-down is paired with a scheduled key-up. Holds default to a 10 s cap (
GAMEPAD_MAX_HOLD_MS) with a 30 s hard cap no caller can exceed — a crashed client can't leave a character running into a wall.release_allalso runs automatically at server shutdown.The HTTP transport refuses to be useful without a token; bind private addresses only. This server is remote control of a keyboard — treat the token like a password.
Where the input lands
HID events go to the frontmost app, full stop. If another app (OBS, a
browser, anything) takes frontmost status, the server still accepts calls
and returns success — the game just never sees the keys. The signature is
unmistakable once you've seen it: tool calls succeed, status shows
held: [], and the character doesn't move. Recovery is to give the game
window input focus again — for a Chiaki stream, click inside the video.
Keymap profiles
Profiles are JSON: DualSense names → key names. See
keymaps/chiaki-ng.json — its button map was read
live from Chiaki's Settings → Keys against a PS5 and verified in game.
The stick entries assume WASD (left) / IJKL (right), but Chiaki ships its
stick bindings unbound: bind Left Stick to W/A/S/D and Right Stick to
I/J/K/L in Chiaki's Settings → Keys first, or hold_stick will do nothing.
Drop your own profiles in ~/.config/gamepad-mcp/keymaps/.
Playing a console through Chiaki-UCR
Chiaki-UCR injects a plugin
(libchiaki_ucr.dylib) into chiaki-ng that presents a virtual DualSense to
the console and maps keyboard/mouse input to it. gamepad-mcp drives the
keyboard side, so the chain is agent → HID keys → UCR → virtual DualSense →
console. Two one-time setup steps and one launch order make it work;
skipping any of them fails silently.

Verified live on 2026-11-21: an agent driving GTA VI on a PS5 through this
chain — buttons first (Circle closed Jason's in-game phone), then sticks
(a 10-second hold_stick walked Jason to the deck railing).
One-time setup: let the plugin load
chiaki-ng 1.10.1's executable is hardened-runtime signed (CodeDirectory
flags 0x10000) with only audio-input and allow-jit entitlements.
UCR's dylib is ad-hoc signed, and without disable-library-validation
macOS silently refuses to load it — UCR looks fine and does nothing.
Re-sign chiaki-ng ad-hoc with an entitlements file
(~/chiaki-entitlements.plist):
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>com.apple.security.device.audio-input</key>
<true/>
<key>com.apple.security.cs.allow-jit</key>
<true/>
<key>com.apple.security.cs.disable-library-validation</key>
<true/>
</dict>
</plist>codesign --force --deep --sign - --entitlements ~/chiaki-entitlements.plist /Applications/chiaki-ng.appReinstalling or upgrading chiaki-ng restores the notarized signature — re-run the re-sign or UCR stops working again.
One-time setup: Input Monitoring
Grant Input Monitoring (System Settings → Privacy & Security → Input Monitoring) to both chiaki-ng and Chiaki-UCR. The plugin's keyboard event tap lives in chiaki-ng's process, so chiaki-ng's grant is the one that makes keys flow.
Launch recipe (verified)
Start from a clean slate, Tailscale up.
Start the gamepad-mcp server in Terminal.
Open Chiaki-UCR and select the game's profile. The selection does not persist across reboots — it defaults to Arc Raiders.
Launch chiaki-ng from UCR ("Abrir chiaki-ng con UCR"). Never launch chiaki-ng directly: a plain running instance blocks the injected one through chiaki-ng's single-instance hand-off.
In chiaki-ng's control plane, double-click the console card to start the stream.
Click inside the stream video to give it input focus.
Play.
When it misbehaves
Stream dropped. chiaki-ng falls back to its control plane (host list; the console shows "State: ready"). The game session persists on the console itself — double-click the console card to reconnect. A click can report a partial-dispatch error and still have worked; verify by the stream actually starting.
The stick readout lies. UCR's on-screen stick telemetry can read +0.00/+0.00 mid-hold while input is working fine. Verify movement by watching the character or scene actually move, never by the readout alone.
Mouse-look is unproven.
mouse_movetakes absolute screen coordinates, and UCR's F8 mouse capture did not pan the in-game camera in testing — an absolute warp moved the macOS cursor only. Sticks are the proven path.
Status
v0.1.2. The controller, keymaps, and MCP protocol are covered by tests that
run anywhere (pip install -e ".[test]" && pytest). The macOS event backend
is exercised on a real Mac; off-Mac it stays behind the backend interface,
with the null backend standing in.
Known issue: the server process has died unexpectedly (twice in one
morning; the port stops answering entirely and clients get an empty
reply). Cause not yet captured — restart with the same command. Every
restart changes the MCP session id, so HTTP clients must re-run
initialize and capture the new mcp-session-id header.
License
Apache-2.0. See LICENSE.
Available Tools
12 toolscomboB
Press several raw keys together (e.g. ['cmd', 'tab']).
Inputs: keys (required), hold_ms.
hold_stickB
Hold an analog stick direction. stick: 'left' or 'right'. direction: up, down, left, right, or a diagonal like 'up-left'. This is how a character walks/runs and how the camera turns.
Inputs: stick (required), direction (required), hold_ms.
key_downB
Hold a raw key down until key_up or the hard cap releases it.
Inputs: key (required).
key_tapC
Tap a raw keyboard key by name (e.g. 'return', 'w', 'esc').
Inputs: key (required), hold_ms.
key_upC
Release a raw key held by key_down.
Inputs: key (required).
mouse_clickC
Click the mouse ('left' or 'right').
Inputs: button, hold_ms.
mouse_moveC
Move the mouse pointer to absolute screen coordinates.
Inputs: x (required), y (required).
pressA
Press a DualSense button: cross, circle, square, triangle, dpad_up/down/left/right, l1, r1, l2, r2, l3, r3, options, share, touchpad, ps. hold_ms is how long the button stays down.
Inputs: button (required), hold_ms.
profilesB
List available keymap profiles.
Inputs: no named parameters.
release_allA
Panic button: release every held key immediately.
Inputs: no named parameters.
set_profileC
Switch the active keymap profile.
Inputs: name (required).
statusB
Backend, permission state, active keymap profile, held keys.
Inputs: no named parameters.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 12 tools
Most tools target clearly distinct device+action pairs (mouse_move vs mouse_click vs press vs key_tap). The only mild overlaps are press vs key_tap (gamepad button vs raw key) and hold_stick vs key_down, but descriptions make the distinction reasonably clear.
Consistent snake_case with verb_* prefixes for action tools (key_tap, key_down, key_up, mouse_move, mouse_click, hold_stick, set_profile). A few bare-noun tools (status, profiles, combo) deviate slightly from the verb_noun pattern but remain readable.
12 tools is well within the sweet spot and each tool covers a distinct input primitive (buttons, sticks, keys, mouse, profiles, panic release). Nothing feels redundant or missing at the count level.
Covers gamepad buttons/sticks, raw keyboard tap/hold/combo, panic release, mouse move/click, and profile management—solid lifecycle coverage. Minor gaps: no mouse scroll or button-down/up for drags, and no explicit stick release beyond the global release_all.
Maintenance
Related MCP Connectors
Operate Linux, macOS and Windows from your LLM. Every action runs through an auditable allowlist.
Give your AI hands on every computer you own. Remote control of your Windows, Mac and Linux machines from one ChatGPT, Claude or Cursor chat — even from your phone. Ask it to find out why the office server is crawling, update every PC in the house, sort out your mom's laptop from another city, or ship the site from your work machine while you're on a train. It runs commands, reads files and takes a screenshot when it needs to see the screen. You decide what each computer allows, and you can cut any of them off in a second.
Connect ChatGPT, Claude, Cursor or VS Code to your Windows PC: files, terminal, browser, apps.
Human-input bridge for AI agents with voice-first answer links, MCP tools, and HTTP APIs.
Related MCP Servers
- FlicenseNot gradedqualityDmaintenanceEnables AI agents to play local Windows games through low-level keyboard/mouse input, screen capture, OCR, and per-game profiles for semantic actions.-
- AlicenseNot gradedqualityCmaintenanceEnables AI agents to operate Windows computers with parallel, human-like input streams, including held keys, concurrent mouse/keyboard actions, UIA/OCR targeting, and a knowledge base for engineering software.1MIT
- AlicenseAqualityBmaintenanceLocal Windows x64 MCP server for USB HID mouse control and optional voice dictation. Requires compatible uMouse hardware, firmware and x64 Node.js 22+. MCPB 0.1.1 packages the legacy 0.1.0 stdio runtime; shared-v2 is a separate source development path. The MCPB includes no cloud credentials, defaults speech recognition to disabled, and supports optional local Windows Speech.22409MIT
- AlicenseNot gradedqualityCmaintenanceEnables agents to control games on Windows through scan-code keyboard presses and holds, relative and absolute mouse movement, clicks, and wheels, timed multi-step input sequences for combos, plus screenshots and window listing/focusing for a see-think-act loop. Input is injected via SendInput so games that only read raw hardware-level input can respond.MIT