# Clara Windows UI MCP

Local-only MCP server for Windows UI automation oriented toward difficult targets such as game windows, legacy desktop apps, and surfaces that do not expose UI Automation reliably.

## Why this exists

The built-in computer-use flow is convenient, but it can struggle with windows that render through game engines or custom graphics pipelines. This MCP takes a lower-level path:

- enumerate and focus windows through `HWND`
- drive mouse and keyboard input through native Win32 APIs
- capture the desktop or a target window to PNG files
- expose the same behavior through MCP stdio and a small local HTTP API

## Current toolset

- `list_windows`
- `get_foreground_window`
- `focus_window`
- `capture_screen`
- `move_mouse`
- `click_mouse`
- `press_key`
- `type_text`

## Vision foundation

The `src/domain/vision` module defines the shared, consumer-facing vision contract:

- `SceneObservation` carries capture provenance, OCR, and detection results.
- `SceneEvent` leaves room for optional temporal change notifications.
- `VisionProfileRegistry` isolates examples, training-set references, and rules by `applicationId`.

Capture and detection implementations are intentionally not part of this foundation. Consumers should depend on the scene/event contract rather than detector internals.

## Vision service and tray

The native projects use the same safe session boundary as the Clara Cursor:

```text
ClaraVisionBroker Windows service -> local named pipe -> ClaraVisionTray user-session agent
```

- The broker persists the high-level observation state (`paused` or
  `observing`) and relays it locally.
- The tray shows that state and lets the user start or pause observation.
- Neither component captures the desktop or performs automation. A future
  session agent will connect the existing opt-in capture pipeline only after a
  target selection and capture policy are defined.

Build `native/ClaraVisionBroker` and `native/ClaraVisionTray` in Release. Then
run `scripts/install-vision-service.cmd` from an elevated Command Prompt and
`scripts/install-vision-tray.cmd` from the interactive user session.

## Run

```powershell
npm.cmd install
npm.cmd run build
npm.cmd run start:stdio
```

For local diagnostics:

```powershell
npm.cmd run start:api
```

## Architecture choices

- run mode: `stdio`, `api`, or `both`
- deployment profile: local-only
- authentication: none
- storage: local file output only for captures
- organization: CQRS-style commands and queries with transport adapters

## Notes

- Screenshots are written under `captures/`.
- This first version avoids UIA on purpose so it remains useful for games and canvas-rendered apps.
- The next logical upgrade is target-aware matching by template image, pixel region, or OCR.