win-pilot

English | 中文

Win-Pilot is an Agent Plugin for Windows Computer Use. It gives Windows agents one consistent computer tool for desktop applications and isolated Chromium sessions, with the same actions, recovery rules, and results across plugin, MCP, CLI, skill, and native-host integrations.

Features

Related MCP server: Clara Windows UI MCP

Requirements

• Windows 10/11 x64

• Node.js 22.12 or newer

• Windows PowerShell 5.1

• Optional .NET 8 runtime for WGC capture

• An Agent Plugin host, stdio MCP client, or shell-capable agent

Quick Start

Install with your Agent (recommended)

Send this single instruction to an agent on the Windows machine:

Install and verify Win-Pilot by following https://raw.githubusercontent.com/JeremyWangCY/win-pilot/main/AGENT_INSTALL.md

The agent will prefer the complete plugin, then fall back to MCP or CLI only when its host lacks plugin support.

Manual installation

Plugin-capable hosts can install JeremyWangCY/win-pilot. The package contains both .mcp.json and skills/win-pilot.

For other hosts:

npm install -g win-pilot@latest
win-pilot doctor --probe

Register this local MCP server through the host's normal plugin/tool settings:

{
  "mcpServers": {
    "win-pilot": { "command": "win-pilot", "args": ["mcp"] }
  }
}

Convenience installers are provided for known hosts, while generic modes remain available for every other agent:

win-pilot install --agent codex
win-pilot install --agent claude
win-pilot install --agent qoder
win-pilot install --agent pi
win-pilot install --agent dsh
win-pilot install --agent mcp
win-pilot install --agent cli

Verify with:

win-pilot --version
win-pilot status --json
win-pilot doctor --probe --json

Expected: version 0.1.1, platform win32, and every required doctor check marked ok.

Updating

Finish active UI tasks, update the runtime, and restart or reload the host's plugin/MCP process:

npm install -g win-pilot@latest
win-pilot doctor --probe

DSH profiles must be restarted separately. Re-run win-pilot install when you want its managed skill copy refreshed; user-maintained custom skills are not updated in the background.

Sandboxed agents

When a sandbox reaps children after every command, configure win-pilot mcp in the host's persistent plugin or MCP supervisor. Do not detach a process from an ephemeral command. Shell-only agents can use one-shot win-pilot request calls.

Usage

MCP and plugin hosts expose computer. Shell-only hosts use the identical JSON interface:

win-pilot request --payload '{"action":"list_apps"}' --json
win-pilot request --payload '{"action":"get_window_state","hwnd":12345,"include_text":true}' --json
win-pilot request --payload '{"action":"click","hwnd":12345,"element_index":7,"snapshot_id":"<snapshot>"}' --json

The live MCP tools/list result is authoritative for the complete parameter schema and defaults.

How it works

Windows Agent
  ├─ Agent Plugin ─┐
  ├─ stdio MCP ────┼─> Win-Pilot runtime ─> PowerShell helper ─> UIA / Win32 / WGC
  ├─ CLI + Skill ──┤          └────────────> CDP provider ─────> isolated Edge
  └─ Native Adapter┘

The runtime is the deep module: it owns action semantics and results. Adapters only register and render its interface, so fixes remain local and every host receives the same behavior.

Security and privacy

Win-Pilot runs with the Windows user's permissions. It contains no telemetry and does not upload screenshots. Browser navigation makes ordinary network requests to requested sites. Installing the plugin does not bypass host approvals, sandbox rules, or task authorization.

Troubleshooting

Compatibility-build diagnostics currently use %TEMP%\win-pilot-diag.log.

Development

git clone https://github.com/JeremyWangCY/win-pilot.git
cd win-pilot
npm test
npm pack --dry-run

Tests exercise the public CLI protocol, MCP discovery/calls, plugin manifest, installation plans, and owned Windows/browser fixtures.

License

MIT