# WinMind

<!-- mcp-name: io.github.jhonpork1233-beep/win-mind -->

[![CI](https://github.com/jhonpork1233-beep/win-mind/actions/workflows/ci.yml/badge.svg)](https://github.com/jhonpork1233-beep/win-mind/actions/workflows/ci.yml)
[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
![Platform: Windows 10/11](https://img.shields.io/badge/platform-Windows%2010%2F11-0078D6.svg)
![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-3776AB.svg)
![MCP](https://img.shields.io/badge/MCP-server-8A2BE2.svg)
![Tools: 222](https://img.shields.io/badge/tools-222-success.svg)

**A Windows MCP server that lets an AI see and drive any app through the accessibility tree - no screenshots, no
mouse, no pixel guessing - plus 200+ native Windows tools.**

Most computer-use agents look at the screen: take a screenshot, send thousands of image tokens to a model, guess
coordinates, click, take another screenshot. WinMind reads what Windows already knows. Every app publishes its
buttons, text boxes, lists and text to UI Automation, the accessibility layer screen readers like Narrator and NVDA
use. WinMind turns that into a compact, numbered tree an AI can read in one call and act on in the next - even when
the app is behind other windows.

```
WhatsApp  (WhatsApp.Root.exe, window)
├─ [6] banner
│  ├─ [7] button "Chats" (42) checked [toggle]
│  ├─ [8] button "Calls" (4) unchecked [toggle]
├─ [20] textbox "Search or start a new chat" [set_value]
├─ [21] tabs "chat-list-filters" (4)
│  ├─ [22] tab "All" selected [select]
│  ├─ [23] tab "Unread 42" [select]
├─ [26] grid "Chat list" (9)
│  ├─ [29] row "Alex Yesterday Photo" [click]
│  ├─ [30] row "Study Group 3 unread messages Yesterday" [click]
└─ [37] button "Send document" [click]
```

`ui_act(ref=30)` opens that chat. `ui_act(name="Play", title_contains="spotify")` plays music. No screenshot was taken.

## Why it's different

| | Screenshot agents | WinMind |
|---|---|---|
| What the model gets | images (thousands of tokens each) | a text tree with real names, states and actions |
| Speed per look | seconds (capture + vision model) | **0.1-0.3 s** for a whole app (one cross-process call) |
| Clicking | pixel coordinates, moves your mouse | the app's own accessibility actions - your mouse never moves |
| Apps behind other windows | must bring them to front | reads and acts in the background |
| Text in editors/documents | OCR | exact text from the app (TextPattern) |

Things WinMind handles that trip up simpler UI Automation tools (details in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)):

- **Chromium-based apps behind other windows** (Discord, Spotify, Steam, Teams, VS Code) expose no content while
  covered - Chromium's occlusion tracking pauses them. WinMind wakes them properly once; the tree then stays readable
  in the background. It never pulls a window over a full-screen game.
- **WinUI 3 apps that host WebView2** (WhatsApp) loop back into their own web content - naive scans fetch the same
  elements thousands of times. WinMind splits and de-duplicates: 7.3 s -> 0.25 s.
- **Web clicks that silently do nothing**: pressing a list item whose click handler sits on an inner link. WinMind
  performs the default action of the element that really owns the click - without moving the mouse.
- **Buttons *and* text in one tree**: editor/document content comes through next to the controls.
- **Finds apps by process**, not just title (Spotify's title is the song playing), including apps hidden in the tray.
- **Token budget**: 222 tool definitions cost ~32k tokens (143 per tool); long outputs are trimmed with a note
  saying what was cut; errors are one actionable line, not tracebacks.

## What's in the box (222 tools)

| Area | Examples |
|---|---|
| **UI automation** | `ui_tree`, `ui_map`, `ui_act`, `ui_read`, `ui_find_text`, `ui_element_at`, `ui_focused` |
| Health & diagnostics | `full_health_scan`, `why_is_my_pc_slow`, `smart_disk_health`, `battery_health`, `reliability_records` |
| Hardware & gaming | CPU/GPU metrics, temperatures, fans, FPS / frame times / stutter / GPU-bottleneck (PresentMon) |
| Live OS tracing | ETW process / DNS / file / network events |
| Network | `why_no_internet`, `port_process_map`, `whats_on_port`, DNS cache, Wi-Fi scan |
| Privacy & security | `whats_using_my_webcam` / `_mic`, Defender, BitLocker, USB history, startup entries |
| Windows & desktops | windows, virtual desktops, saved workspaces |
| Files, processes, services, registry | with guards against system folders, critical keys and core OS processes |
| Display, audio, media, speech | per-app volume, now-playing + media control, brightness, night light, TTS |
| Screen capture (explicit only) | `capture_window_image` renders one window even when it's covered |

Full list with one line per tool: [TOOLS.md](TOOLS.md). An AI can also call `tool_guide` for a categorized index.

## Quick start

### One-click: Claude Desktop

Download `winmind-mcp-<version>.mcpb` from [Releases](https://github.com/jhonpork1233-beep/win-mind/releases) and
open it - Claude Desktop installs it like a browser extension (it needs [uv](https://docs.astral.sh/uv/), which sets up
the Python dependencies for you). The bundle includes the native helpers.

### Any MCP client (Claude Code, Cursor, VS Code, local models...)

Requirements: Windows 10/11, Python 3.10+.

```powershell
git clone https://github.com/jhonpork1233-beep/win-mind
cd win-mind
pip install -e .
```

Add it to your MCP client. Use the **full path** to Python so a changed PATH can never break it:

```jsonc
// Claude Desktop / Claude Code (~/.claude.json "mcpServers") / any MCP client
{
  "winmind": {
    "command": "C:\\Path\\To\\Python\\python.exe",
    "args": ["-m", "winmind"]
  }
}
```

Claude Code shortcut: `claude mcp add winmind -- "C:\Path\To\Python\python.exe" -m winmind`

Then ask things like *"what's in my WhatsApp unread chats?"*, *"play my liked songs on Spotify"*, *"why is my PC
slow?"*, *"screenshot Notepad without switching windows"*.

### Optional native helpers

The core (UI engine, files, processes, registry, health, network...) is pure Python. Some tools use small native
helpers - live event hooks, ETW, window streaming, virtual desktops, the WinRT broker (media, notifications), HUD
overlays, PresentMon. Build them from source in `native/`:

```powershell
powershell -ExecutionPolicy Bypass -File scripts\build_all.ps1
```

(Visual Studio 2022 Build Tools with the C++ workload and Windows SDK; MinGW `g++` for `winmind_native.dll`.) Tools
whose helper isn't built say so and name the script to run.

## Using it efficiently (for AI clients and prompt writers)

- Start with `ui_tree` (structure) or `ui_read` (just the text) - not screenshots.
- Act by name when it's unambiguous: `ui_act(name="Search", title_contains="spotify")`; by `ref` otherwise.
- Prefer summary tools (`full_health_scan`, `system_health_summary`, `windows_summary`) over raw listings
  (`appx_packages`, `service_list`, `port_process_map`).
- Every tool accepts `detail=true` (no trimming), `compact=true` (short keys).
- Clients with deferred tool loading (e.g. Claude Code's tool search) only load the definitions they use.

## Safety

WinMind runs with your user's rights and can change your system - that's its purpose. Read [SECURITY.md](SECURITY.md).

- Every tool carries MCP risk annotations (`readOnlyHint`, `destructiveHint`) so clients can ask before risky calls;
  16 tools are marked destructive.
- Built-in guards: file tools refuse Windows/System32/Program Files and drive roots; registry writes/deletes refuse
  critical keys; process tools refuse core OS processes.
- The UI engine never takes screenshots and never moves the mouse. Screen capture exists only as explicit tools.
- No telemetry. Everything WinMind writes stays in `%LOCALAPPDATA%\WinMind` (override with `WINMIND_HOME`).
- Run it non-elevated unless you need admin-only tools.

## Testing

```powershell
pip install -e .[dev]
python tests\audit_tools.py tests\audit.json     # calls every safe tool, sandbox-tests the write tools
python tests\audit_summary.py tests\audit.json
```

The audit only writes to a temp folder, the throwaway registry key `HKCU\Software\WinMindAudit`, a process it starts
itself and its own Notepad window. Current results: 102/109 read-only tools OK (the rest need admin rights, a
running session or an open Office document), 37/37 sandbox and guard tests pass.

## Roadmap

- Java Access Bridge (Java/Swing apps), IAccessible2 text for Firefox/LibreOffice
- Stable selectors and recorded workflows
- Live UI event stream (tree deltas instead of re-reading)
- UI Automation Remote Operations for even faster trees
- One-click installs: Claude Desktop extension, Windows agent connector (On-Device Registry)
- Public head-to-head benchmark against other Windows MCP servers

## License

[Apache-2.0](LICENSE). Third-party components: [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).