Skip to main content
Glama
Windows Computer Use MCP

Windows Computer Use MCP

by cooolie

Windows Computer Use MCP

tests License: MIT

A native Windows MCP server for desktop automation. It lets an MCP client inspect and control ordinary Windows applications through UI Automation, SendInput, and GDI screen capture — no macOS app bundle, Node runtime, or vendor-specific agent required.

Features

  • Accessibility-first: every get_app_state returns a compact UI Automation tree with element indices, roles, names, values, bounds, and supported actions.

  • Ten tools for window discovery, snapshots, clicking, typing, key presses, scrolling, dragging, text selection, value setting, and accessibility actions.

  • State-safe element actions: element indices are tied to the state_id of the snapshot that produced them. Bounds, enabled state, and offscreen state are re-read when an action runs, so a moved or vanished control fails instead of clicking a stale location.

  • DPI and multi-monitor aware: windows, screenshots, and input use absolute physical screen pixels across the virtual desktop.

  • Safety checks: UIPI integrity preflight, top-level window hit testing, cursor-movement verification, out-of-desktop coordinate rejection, and a read-only mode that hides all input tools.

  • Small dependency footprint: the only runtime dependency is comtypes; PNG encoding is implemented with the standard library.

Related MCP server: Deskhand-mcp

Requirements

  • Windows 10 or Windows 11, 64-bit.

  • Python 3.10 or newer.

  • comtypes for UI Automation. Without it, list_apps, screenshots, and coordinate input still work; the accessibility tree is unavailable.

The server does not require administrator rights. It checks process token integrity before sending input and refuses targets at a higher integrity level (Windows UIPI). It cannot operate on the secure desktop (UAC prompt or lock screen).

Install

git clone https://github.com/cooolie/windows-computer-use-mcp.git
cd windows-computer-use-mcp
python -m pip install -e .

Verify the test suite:

python -m unittest discover -s tests -v

Configure an MCP client

Add a stdio server entry to your MCP client's configuration. For Claude Code, the user-scope configuration lives in the top-level mcpServers object of ~/.claude.json:

{
  "mcpServers": {
    "computer-use-windows": {
      "type": "stdio",
      "command": "C:\\path\\to\\python.exe",
      "args": ["-m", "computer_use_windows"],
      "env": {
        "PYTHONUNBUFFERED": "1",
        "PYTHONIOENCODING": "utf-8",
        "PYTHONUTF8": "1"
      },
      "timeout": 120000
    }
  }
}

Use the absolute path to the Python interpreter where the package is installed (for example, a virtual environment's python.exe). Claude Code inherits only a small allowlist of environment variables on Windows, so a bare python may not resolve to the intended interpreter.

To run directly from a checkout without installing the package, install the dependency and point the client at the launcher instead:

python -m pip install -r requirements.txt
{
  "command": "C:\\path\\to\\python.exe",
  "args": ["C:\\path\\to\\windows-computer-use-mcp\\bin\\computer-use-mcp-server.py"]
}

The server also includes .claude-plugin/plugin.json if you package it in a Claude Code plugin marketplace.

Optional Claude Code skill

The repository ships a skill that teaches Claude Code when and how to use the tools. Copy it into your personal skills directory:

Copy-Item skills\computer-use\SKILL.md `
  "$HOME\.claude\skills\computer-use-windows\SKILL.md"

Restart Claude Code. The tools appear as mcp__computer-use-windows__*.

Tools

Tool

Purpose

list_apps

List visible top-level windows with process, title, class, hwnd, and bounds.

get_app_state

Activate a window, return its accessibility tree, and capture a screenshot.

click

Click an element center or absolute screen coordinates with optional modifiers.

set_value

Set a value through ValuePattern, falling back to focus + select-all + typing.

type_text

Type Unicode text into the focused control.

press_key

Press a key or combination such as Return, ctrl+s, or alt+f4.

scroll

Rotate the mouse wheel at an element or coordinate.

drag

Drag between two absolute screen coordinates.

select_text

Select matching text inside a TextPattern element.

perform_secondary_action

Invoke an action the element explicitly advertises.

Typical loop

  1. list_apps to find the target window.

  2. get_app_state with the app name, title, process, or hwnd. Record the returned state_id.

  3. Act on an element_index from that state, passing the same state_id.

  4. Call get_app_state again before deciding the next action.

All element indices and state_id values expire on the next snapshot.

Coordinates and screenshots

  • Element rectangles and input coordinates are absolute physical screen pixels on the virtual desktop. Multi-monitor coordinates may be negative.

  • Screenshots may be scaled down. The result reports origin and scale, where scale is image pixels divided by screen pixels: screen = origin + image_pixel / scale.

  • A screenshot captures the visible screen rectangle, not an offscreen window surface. If the target cannot be brought to the foreground, the screenshot is omitted rather than capturing an unrelated window.

  • UI Automation trees from Chromium/Electron applications are often sparse; use the screenshot and coordinates for those apps.

Safety and limits

  • The server targets existing windows; it does not launch applications. Start the app with the normal shell or CLI first.

  • Set COMPUTER_USE_WINDOWS_READ_ONLY=1 to expose only list_apps and get_app_state. In read-only mode get_app_state does not activate the window and captures no screenshot unless the target is already foreground.

  • Screenshots can contain sensitive data. Do not send them to external services or include them in reports without the user's approval.

  • Input is refused when the server or target integrity level cannot be read, or when the target is elevated above the server.

  • Cloaked windows are excluded from list_apps; minimized windows are listed with minimized=yes and get_app_state restores them.

  • The server is single-threaded and handles one tool call at a time.

See SECURITY.md for the security policy and private reporting instructions.

Project layout

bin/computer-use-mcp-server.py   dependency-free checkout launcher
computer_use_windows/
  server.py                      MCP stdio loop and protocol handling
  schemas.py                     tool definitions and JSON schemas
  tools.py                       dispatch and tool implementations
  uia.py                         UI Automation snapshots and actions
  win32.py                       ctypes bindings
  apps.py                        window enumeration and activation
  input.py                       SendInput keyboard and mouse injection
  capture.py                     GDI screen capture
  keys.py                        key-combination parser
  png.py                         dependency-free PNG encoder
  geometry.py                    rectangle helpers
skills/computer-use/SKILL.md     optional Claude Code skill
tests/                           unit and stdio integration tests

Development

python -m pip install -e .
python -m unittest discover -s tests -v

The tests run on Windows and do not inject mouse or keyboard input. The stdio integration test starts the server, performs the MCP handshake, lists tools, and calls list_apps. GitHub Actions runs the suite on Python 3.10 and 3.12.

License

MIT. See LICENSE.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      C
      maintenance
      Enables MCP clients to observe and control Windows desktop applications by reading UIA element trees and capturing screenshots, then acting through UIA patterns or synthesized mouse and keyboard input. It runs standalone on an interactive Windows session, without requiring the original host application.
      1
      -
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables an MCP host to observe and control Windows desktop applications deterministically: reading UIA control trees, OCR text and pixel diffs, clicking, typing, scrolling and window management, plus verbatim text reading/appending via ValuePattern/TextPattern. Visual judgment is delegated back to the host model's own multimodal context through annotated screenshots, numbered element cards, grid maps and crosshair zoom-ins, so no separate vision model is needed.
      1
      AGPL 3.0
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables MCP clients on Windows 10/11 to control an interactive desktop through screenshots, mouse and keyboard actions, window management, UI Automation, clipboard access, and multi-display switching. Supports batched actions, application access controls, and visual or text-based desktop interaction.
      50 npm
      1
      MIT