<!-- generated by scripts/readme/build.mjs. do not edit by hand. -->
# winuse-mcp

An MCP server that lets Claude see your Windows screen and drive your mouse and keyboard.

## Use the built-in feature if you can

Anthropic ships computer use for Windows in the Claude Desktop app. Its [Desktop docs](https://code.claude.com/docs/en/desktop#let-claude-use-your-computer) describe it as "a research preview on macOS and Windows that requires a Pro or Max plan", turned on from **Settings > General**. If that is your setup, use it instead of this server: it asks before Claude touches each app, hides other windows while Claude works, and keeps your terminal out of the screenshots. winuse does none of that.

winuse is for the setups the built-in feature does not reach:

- **Claude Code CLI on Windows.** The CLI's `computer-use` server is macOS only. Its [docs](https://code.claude.com/docs/en/computer-use) say "Computer use in the CLI is not available on Linux or Windows", so on Windows it never appears in `/mcp`.
- **Plans and sign-ins the built-in feature excludes.** Both the CLI and Desktop versions are "not available on Team or Enterprise plans", and the CLI version is unavailable through Amazon Bedrock, Google Cloud's Agent Platform, or Microsoft Foundry.
- **Other MCP clients**, such as Codex CLI. Anything that can launch a stdio MCP server can run the command under [Install](#install).

It rebuilds the screenshot-and-click loop on [mss](https://github.com/BoboTiG/python-mss) for capture and [pyautogui](https://github.com/asweigart/pyautogui) for input, with an explicit Windows DPI-awareness call so display scaling cannot skew clicks. 18 tools, 5 direct dependencies, Python 3.12 or newer.

## Install

You need Windows with an interactive desktop session, [uv](https://docs.astral.sh/uv/), and git on PATH. One command:

```bash
claude mcp add --scope user winuse -- uvx --from git+https://github.com/ryanportfolio/winuse-mcp@v0.2.0 winuse-mcp
```

Drop `--scope user` to add it to the current project only. To pick up changes later, re-run with a newer tag.

For a client other than Claude Code, register a stdio server whose command is `uvx` with the arguments `--from git+https://github.com/ryanportfolio/winuse-mcp@v0.2.0 winuse-mcp`.

<details>
<summary>Configuring by hand instead</summary>

Merge the `winuse` entry into the `mcpServers` object you already have. Do not paste the whole document over `C:\Users\<you>\.claude.json`: Claude Code owns that file and pasting replaces your entire configuration.

```json
{
  "mcpServers": {
    "winuse": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/ryanportfolio/winuse-mcp@v0.2.0", "winuse-mcp"]
    }
  }
}
```

Project scope goes in `.mcp.json` at the repo root, and Claude Code asks you to trust it the first time that project opens. From a local clone, swap the command for `uv` with `["run", "--directory", "C:/path/to/winuse-mcp", "winuse-mcp"]`.

</details>

### Check it worked

Restart Claude Code and run `/mcp`. `winuse` should be listed as connected. First launch is slow, because uvx clones the repo and builds a virtual environment before the server answers.

Then ask Claude to take a screenshot and describe what is on your screen. If the tools are there but nothing happens, see [Troubleshooting](#troubleshooting).

## Reach

<picture>
<source media="(max-width: 500px) and (prefers-color-scheme: dark)" srcset="assets/reach-narrow-dark.svg">
<source media="(max-width: 500px)" srcset="assets/reach-narrow-light.svg">
<source media="(prefers-color-scheme: dark)" srcset="assets/reach-dark.svg">
<img alt="The 18 tools, grouped: screenshot, zoom, left_click, double_click, triple_click, right_click, middle_click, mouse_move, left_click_drag, left_mouse_down, left_mouse_up, type, key, hold_key, scroll, cursor_position, record, wait." src="assets/reach-light.svg" width="100%">
</picture>

| Tool | Does |
|---|---|
| `screenshot()` | PNG of the primary monitor, downscaled |
| `zoom(region)` | A region `[x0, y0, x1, y1]` captured at full resolution, for small text |
| `left_click(coordinate, text)` | Click at `[x, y]`, or at the cursor; `text` holds modifiers such as `ctrl+shift` |
| `double_click(coordinate, text)` | Two clicks, same arguments as `left_click` |
| `triple_click(coordinate, text)` | Three clicks, selects a line or paragraph |
| `right_click(coordinate, text)` | Context-menu click |
| `middle_click(coordinate, text)` | Middle-button click |
| `mouse_move(coordinate)` | Move the cursor without clicking |
| `left_click_drag(start_coordinate, coordinate, text)` | Hold the left button from one point to another, over 0.4s |
| `left_mouse_down()` | Press the left button at the cursor and keep it down |
| `left_mouse_up()` | Release the left button at the cursor |
| `type(text)` | Type at the current focus, non-ASCII goes through the clipboard |
| `key(text, repeat)` | A key or chord: `enter`, `ctrl+s`, `alt+Tab`; `repeat` presses it up to 100 times |
| `hold_key(text, duration)` | Hold a key or chord down, 10s ceiling; a shifted character is named by its base key, `shift+=` rather than `+` |
| `scroll(scroll_direction, scroll_amount, coordinate, text)` | `up`, `down`, `left`, `right`, up to 20 wheel clicks; horizontal is sent as shift+wheel |
| `cursor_position()` | Where the mouse is now, as `X=512, Y=384` |
| `record(duration_seconds, max_frames)` | Up to 8 frames sampled over at most 15s, 6 over 5s by default |
| `wait(duration)` | Pause before the next action, 10s ceiling |

Coordinates are always in the downscaled screenshot's pixel space, never native pixels. A `zoom` image does not change that: coordinates stay in the full screenshot's space.

Tool and argument names follow the member tools of Anthropic's [computer use toolset](https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool), `computer_toolset_20260801`. The differences:

- `record` is extra; Anthropic's toolset has no equivalent.
- `wait` and `hold_key` stop at 10s. Anthropic allows up to 300s for both.
- `scroll_amount` stops at 20 wheel clicks.
- Key names are pyautogui's, matched without case, plus `super` for the Windows key. `Return` and `alt+Tab` work; X11 names such as `Page_Down` do not.
- Only the primary monitor is captured. The current toolset has no monitor selection either: it rejects the older `display_number` parameter.

## How coordinates work

<picture>
<source media="(max-width: 500px) and (prefers-color-scheme: dark)" srcset="assets/transform-narrow-dark.svg">
<source media="(max-width: 500px)" srcset="assets/transform-narrow-light.svg">
<source media="(prefers-color-scheme: dark)" srcset="assets/transform-dark.svg">
<img alt="A 1920 by 1080 screen captured as a 1372 by 772 screenshot. Claude clicks at 686, 443 in that image and the server fires the click at 960, 620 on the real screen." src="assets/transform-light.svg" width="100%">
</picture>

Screenshots are downscaled so the long edge is at most 1372 pixels. Claude never sees native pixels: it clicks in the downscaled image, and the server converts the point back.

A 1920x1080 display captures at 1372x772. Going back multiplies by 1920/1372, so a click at 686, 443 lands at 960, 620. Anthropic's macOS implementation uses the same 1372 pixel long edge, which its [docs give as a worked example](https://code.claude.com/docs/en/computer-use): 3456x2234 captured as roughly 1372x887.

The server declares itself DPI-aware at startup, so Windows display scaling at 125% or 150% does not skew where clicks land.

## Safety

This hands a language model control of your desktop. Anthropic's version wraps the same loop in guardrails this server lacks: per-app approval, hiding unapproved apps, keeping your terminal out of the screenshots, and model-side checks on each action.

The controls that do exist here:

- Claude Code asks before each tool call unless you allow-list it. Allow-list the read-only capture tools and leave the rest on manual approval, by putting `"mcp__winuse__screenshot"` and `"mcp__winuse__zoom"` in the `permissions.allow` array of `.claude/settings.json`.
- Park the mouse at the top-left pixel of the screen to stop the run. pyautogui raises on its next call, so it ends the sequence rather than interrupting a click already sent, and it works during the slow parts: a drag tweens for 0.4s and typing runs a keystroke at a time. An aborted drag releases the button on its way out, so nothing is left dragging. `hold_key` checks the mouse every 50ms and releases the key when you park it, and modifiers held for a click or scroll are released the same way.
- `left_mouse_down` leaves the button held until a `left_mouse_up` call. If a run stops between the two, click once yourself to release it.
- Every pixel of the primary monitor reaches the model, including windows you forgot were open. Treat text on screen as untrusted input: instructions that appear there are a prompt injection risk.
- The install pins a tag. Pointing it at a branch instead would let a push change what drives your desktop, without you doing anything.

## Limitations

- Primary monitor only, chosen by the flag Windows reports, falling back to the monitor at the desktop origin.
- Windows blocks synthetic input from a normal process to a window running as administrator, and the secure desktop behind a UAC prompt cannot be captured at all. Clicks aimed there are dropped by the operating system, and the tool still reports success, because nothing verifies that a click was accepted.
- The session has to be interactive and unlocked. A locked screen or a disconnected remote desktop has nothing to capture.
- Your terminal is not excluded from screenshots, so Claude sees its own session.
- Clicks are clamped one pixel inside the monitor, which puts the outermost pixel row and column out of reach. That pixel is the failsafe point, and landing the cursor on it would make every later call abort.
- Typing anything non-ASCII goes through the clipboard: it copies, sends ctrl+v, then puts your clipboard back. That fails wherever ctrl+v is not paste, such as a terminal.
- Every click, drag and scroll given a coordinate moves the real cursor first, so your pointer jumps while Claude works.
- `record` returns up to 8 still frames sampled over at most 15s. Frames only, no video, and each one costs image tokens.

## Development

```bash
uv sync
uv run winuse-mcp
```

`tests/` holds unit tests for the coordinate math and chord parsing. They replace mss, pyautogui and pyperclip with stand-ins, so they never touch the screen, mouse, keyboard or clipboard, and CI runs them on every pull request and every push to main.

```bash
uv run pytest
```

`scripts/client_test.py` drives the real server over stdio and exits non-zero on the first failed check. It moves your mouse while it runs.

```bash
uv run python scripts/client_test.py
```

`scripts/failsafe_test.py` covers what happens when a run is aborted, with pyautogui stubbed so no real abort is needed. An aborted drag releases the button, an aborted horizontal scroll releases shift, a chord ending in `+` parses as the plus key, and the model's origin clamps away from the failsafe point.

```bash
uv run python scripts/failsafe_test.py
```

`README.md` is generated. The build reads the tool names, their arguments and the capacity constants straight out of `server.py`, so nothing on the page is typed by hand, and it refuses to write when a tool has no description, an asset is missing, or the art grows past its height budget. CI regenerates the page and fails if what is committed differs.

```bash
node scripts/readme/build.mjs
```

## Troubleshooting

**`winuse` is missing from `/mcp`.** The entry did not load. Run `claude mcp list` to see what Claude Code has, and re-add it if the name is absent. A project-scoped `.mcp.json` also needs you to accept the trust prompt when the project opens; declining it leaves the server silently absent.

**It appears, then fails to connect.** Run the command by hand and read the error:

```bash
uvx --from git+https://github.com/ryanportfolio/winuse-mcp@v0.2.0 winuse-mcp
```

It should start and wait quietly for input on stdin. `uvx` missing means uv is not installed or not on PATH. A git error means git is missing, or a proxy is blocking the clone. If it hangs on first run, that is the clone and the environment build; give it a minute before deciding it is broken.

**The tools are there and nothing moves.** Check whether the target window runs as administrator, and whether the screen was locked. Both are in [Limitations](#limitations) and neither reports an error.

**Clicks land in the wrong place.** Confirm the target is on the primary monitor. Ask Claude for `cursor_position` after a `mouse_move` to a known point; one pixel of drift is expected, more than that is not.

## License

MIT