RelayMCP

RelayMCP lets your AI agent operate your Windows handheld. Your ROG Ally, Legion Go, or similar device becomes a set of MCP tools on your laptop. GitHub Copilot CLI, Claude Code, VS Code, or any MCP client can then see the screen, tap, type, press controller buttons, run PowerShell, change settings, and talk back out loud. You can also hold two buttons on the handheld and just ask.

you ▸ copilot -p "Open Minecraft on my Ally, and once it's on the title screen, set brightness to 40%"
      ● ally Screenshot   ● ally App (launch Minecraft)   ● ally WaitFor   ● ally-handheld set_brightness
      Minecraft is on the title screen and brightness is at 40%.

What you get

• 🖥️ Computer use. Screenshots, UI-tree snapshots, clicks, typing, app launching, PowerShell, files and the registry, via Windows-MCP.

• 🎮 Handheld hardware (35 tools):

  • a virtual Xbox controller (games see it), plus reading the real one and rumble

  • real multi-touch (tap, swipe, pinch), scan-code keys that work in games, mouse-look

  • speakers, microphone, system-audio capture, brightness, resolution/refresh rate, power mode, battery and CPU load

• 🗣️ Voice prompts. Hold View + Menu (or tap Ask Copilot) and speak. The handheld transcribes locally (Whisper), your agent does the work, and the reply is spoken in a natural neural voice (Kokoro, also local).

• 🏠 Home-only by design. Everything switches on only on your home network, recognized by your router's hardware address. On any other network, or offline, nothing listens and nothing phones home. Normal handheld use is never affected.

• 🔐 SSH underneath. Key-only OpenSSH, local-subnet firewall, MCP servers bound to 127.0.0.1 on the handheld and reached only through the tunnel. See the security model.

• 🧰 Zero-typing handheld setup. One command on your computer builds a kit. Double-tap it on the handheld (from a USB stick, or with a one-liner) and you're done. A Repair RelayMCP shortcut fixes anything later.

• ☕ Keep-awake that respects your battery. The handheld stays awake only while an agent is using it (or for as long as you ask), then sleeps normally.

Related MCP server: WinCommander

Quick start

You need a Windows 11 handheld (tested on an original ROG Ally) and a Mac, Linux or Windows computer on the same home network, with uv and OpenSSH. Voice prompts use GitHub Copilot CLI by default.

1. Install RelayMCP on your computer

uv tool install git+https://github.com/brennengreen/RelayMCP

2. Run setup (it asks two questions, then builds the handheld's kit and waits for it to check in)

relaymcp setup

3. On the handheld (signed in, at home), do either of these, then choose Yes when Windows asks:

• copy the kit folder it printed to a USB drive and double-tap Setup RelayMCP.cmd, or

• press Win+R and run the one-liner it printed: powershell -c "irm http://<your-computer>:8766/<token> | iex"

The first run takes a few minutes: it downloads Python, the speech models and the voice. When the handheld checks in, relaymcp setup trusts its SSH key, starts the tunnel and runs a health check:

  ✓ Tunnel (tools): up 192.168.1.23
  ✓ MCP 'ally': http://127.0.0.1:8765/mcp (20 tools)
  ✓ MCP 'ally-handheld': http://127.0.0.1:8767/mcp (35 tools)
  ✓ Voice dispatcher: agent=copilot permissions=handheld
  ✓ Device agent: windows-mcp up (restarts 0), hardware up (restarts 0)

Then try it:

copilot -p "Take a screenshot of my handheld and tell me what's on screen"
relaymcp say "Hello from RelayMCP"

To use other MCP clients, run relaymcp mcp --print for ready-made config (Claude Code, VS Code, generic JSON). The full walkthrough is in docs/getting-started.md.

How it works

flowchart LR
  subgraph PC["Your computer (macOS · Linux · Windows)"]
    A["AI agent<br/>Copilot CLI · Claude Code · VS Code"] -->|MCP over HTTP| T["relaymcp daemon<br/>SSH tunnels"]
    V["voice dispatcher"] --> A
  end
  subgraph H["Windows handheld (at home only)"]
    S["sshd<br/>key-only · LAN-only"]
    G["RelayMCP agent (hidden)"] --> W["Windows-MCP<br/>127.0.0.1:8765"]
    G --> HW["hardware server<br/>127.0.0.1:8767"]
    C["controller (SYSTEM)<br/>home/away switch"] -.starts/stops.-> S & G
  end
  T <-->|"SSH: -L 8765, -L 8767, -R 8768"| S
  HW -->|"push-to-talk prompts"| V

• On your computer: relaymcp daemon runs as a background service (launchd, systemd or a logon task). It keeps two SSH connections open while the handheld is reachable. One forwards the MCP ports to your 127.0.0.1; the other reverse-forwards the voice dispatcher to the handheld.

• On the handheld: a SYSTEM controller task checks the network at boot, at login, on every network change and every 15 minutes. It runs SSH and the agent only at home. The agent is one windowless pythonw process that runs both MCP servers with hidden consoles, restarts them if they crash, and keeps the device awake while it's in use. There are no cmd/conhost tricks, so antivirus heuristics have nothing to object to.

The details (what's installed where, the job-object lifecycle, and the network rules) are in docs/how-it-works.md.

Voice prompts

Hold View + Menu for about a second (or tap Ask Copilot, or press Ctrl+Alt+Shift+F12). After a chime, speak. Recording stops when you do. Say "new conversation" to start fresh.

By default, voice requests can only use the handheld's tools. relaymcp voice --permissions full also lets them run things on your computer. Change the voice with relaymcp voice --voice am_michael. More in docs/voice.md.

Command reference

Documentation

• Getting started: the full setup walkthrough, updating, uninstalling

• How it works: architecture, lifecycle, and file locations

• Security model: what's exposed, to whom, and when

• Voice prompts: triggers, models, settings, custom agents

• MCP tools: every hardware tool and its parameters

• Configuration: ~/.relaymcp/config.json reference

• Troubleshooting: unreachable device, new router, reset handheld, antivirus

• Development: repo layout, tests, and the device dev loop

Status

RelayMCP is young (v0.1). It is developed on macOS with an original ROG Ally (Windows 11), where everything above is tested end to end. Linux and Windows hosts use the same code paths (OpenSSH, systemd user services, scheduled tasks) and are covered by CI, but have seen less real-world use. Other Windows handhelds should work; the Armoury Crate niceties are ROG-specific. Issues and PRs are welcome.

Acknowledgements

RelayMCP builds on Windows-MCP (screen control), faster-whisper (speech-to-text), Kokoro via kokoro-onnx (voice), ViGEmBus and vgamepad (virtual controller), Win32-OpenSSH, and the MCP Python SDK.

License

MIT. Note that the optional neural voice pulls in GPL-licensed phonemizer/espeak-ng packages on the handheld; see docs/voice.md.