# dsh-computer-use-mcp

English | [中文](README.zh.md)

Standalone [Model Context Protocol](https://modelcontextprotocol.io/) server that exposes desktop GUI automation — "computer use" — for **macOS and Windows**. Any MCP host can drive it: [DeepSeek Harness (dsh)](#deepseek-harness-dsh), Claude Code, Cursor, and every other MCP client.

One tool, `computer_use`, with 23 actions: accessibility-tree snapshots with numbered refs, click / drag / type (CJK via clipboard) / hotkeys, app launch & focus, menu selection, screenshot artifacts, and a Chrome DevTools Protocol fast path for browser tabs.

## Highlights

- **Accessibility-first** — snapshots return a numbered tree; every subsequent action addresses elements by ref, not by brittle coordinates.
- **Cross-platform** — macOS (AX API via AppleScript/JXA) and Windows (UIA3 COM fast path with managed fallback).
- **Browser CDP fast path** — Chrome-family targets automatically switch to a DevTools session for sub-second snapshots and occlusion-proof input (`navigate` / `read_page` / `js_eval` / `tabs`).
- **Screenshots as image artifacts** — vision models receive PNGs; files persist under `CU_HOME` for audit.
- **Safety rails built in** — a static danger-guard intercepts dangerous script payloads; per-app "always allow" grants persist in an owned table.

## Install

Node ≥ 20.

```sh
npm install -g dsh-computer-use-mcp   # then run: dsh-computer-use-mcp
# or one-shot:
npx -y dsh-computer-use-mcp
```

## Hook it to your host

### DeepSeek Harness (dsh)

dsh bridges MCP servers through [`@deepseek-ai/dsh-mcp-client`](https://github.com/deepseek-ai/deepseek-harness). Pass the ready-made overlay in [`dsh/computer-use.cordis.yml`](dsh/computer-use.cordis.yml):

```sh
dsh --profile <name> --patch /path/to/dsh-computer-use-mcp/dsh/computer-use.cordis.yml
```

The model sees the tool as `mcp__computer-use__computer_use`. See [dsh/README.md](dsh/README.md) for the smoke tests that prove the integration keyless.

### Claude Code

```sh
claude mcp add computer-use -- npx -y dsh-computer-use-mcp
```

### Cursor / generic `mcpServers` JSON

```json
{
  "mcpServers": {
    "computer-use": { "command": "npx", "args": ["-y", "dsh-computer-use-mcp"] }
  }
}
```

## The `computer_use` tool

Single tool, dispatched by `action`. Full JSON schema ships on `tools/list`.

| Group | Actions |
|---|---|
| Read / probe | `check_permissions`, `list_apps`, `snapshot`, `find`, `wait_for`, `wait` |
| Input | `click`, `double_click`, `right_click`, `scroll`, `drag`, `type`, `set_value`, `key`, `paste_text` |
| App / window | `focus_app`, `launch_app`, `menu_select` |
| Browser (CDP) | `navigate`, `read_page`, `js_eval`, `tabs`, `browser_adopt` |

Typical loop: `snapshot` → reason over the numbered tree → act by ref (`click {ref}`) → the post-action feedback re-snapshots and reports the UI diff.

## Security model — read before enabling

- **MCP has no approval concept**: a tool call executes immediately. Enable your host's tool-approval / confirmation layer; do not grant blanket autonomy to untrusted prompts.
- **danger-guard** statically intercepts dangerous script payloads inside the implementation.
- **Per-app "always allow" grants** persist under `CU_HOME` (default `~/.tianshu-computer-use/`), separate from any host's own permission store.
- **macOS**: grant the terminal (or host app) running this server both *Accessibility* and *Screen Recording* permissions. The `check_permissions` action self-probes and reports what is missing.

## Environment variables

| Variable | Effect |
|---|---|
| `CU_HOME` | Data directory for grants and screenshot artifacts (default `~/.tianshu-computer-use`) |
| `CU_SESSION_ID` | Artifact subdirectory name per session (default `mcp`) |
| `RIVET_CU_CDP=0` | Disable the browser CDP fast path |
| `RIVET_CU_FEEDBACK=0` | Disable post-action UI feedback summaries |
| `RIVET_CU_HOST=0` | Disable the script host (macOS JXA / Windows PowerShell channel) |
| `RIVET_CU_COM=0` | Disable the Windows UIA3 COM fast path (falls back to the managed builder) |

## Development

```sh
npm install
npm run build                      # tsup → dist/index.js
npm run smoke                      # SDK-client stdio smoke (tools/list + wait)
node integration/verify-server.mjs # zero-dependency JSON-RPC smoke
```

`integration/dsh-smoke.ts` is a keyless integration smoke against DeepSeek Harness's real plugin composition — run it from a [deepseek-harness](https://github.com/deepseek-ai/deepseek-harness) checkout ([instructions](dsh/README.md)).

Extracted from [huiliyi37/Tianshu-harness](https://github.com/huiliyi37/Tianshu-harness) — the tianshu (天枢) harness repository. The implementation under `impl/` is a frozen snapshot of its `computer-use` module; this repository is now that code's source of record.

## License

[GNU AGPL-3.0](LICENSE) — source-available with strong copyleft: modifications
distributed or served over a network must be shared. Local use from any MCP host
is unrestricted. Commercial licensing available on request.