dsh-computer-use-mcp

English | 中文

Standalone Model Context Protocol server that exposes desktop GUI automation — "computer use" — for macOS and Windows. Any MCP host can drive it: DeepSeek Harness (dsh), Claude Code, Cursor, and every other MCP client.

One tool, computer_use, with 23 actions: accessibility-tree snapshots with numbered refs, click / drag / type (CJK via clipboard) / hotkeys, app launch & focus, menu selection, screenshot artifacts, and a Chrome DevTools Protocol fast path for browser tabs.

Highlights

• Accessibility-first — snapshots return a numbered tree; every subsequent action addresses elements by ref, not by brittle coordinates.

• Cross-platform — macOS (AX API via AppleScript/JXA) and Windows (UIA3 COM fast path with managed fallback).

• Browser CDP fast path — Chrome-family targets automatically switch to a DevTools session for sub-second snapshots and occlusion-proof input (navigate / read_page / js_eval / tabs).

• Screenshots as image artifacts — vision models receive PNGs; files persist under CU_HOME for audit.

• Safety rails built in — a static danger-guard intercepts dangerous script payloads; per-app "always allow" grants persist in an owned table.

Install

Node ≥ 20.

npm install -g dsh-computer-use-mcp   # then run: dsh-computer-use-mcp
# or one-shot:
npx -y dsh-computer-use-mcp

Hook it to your host

DeepSeek Harness (dsh)

dsh bridges MCP servers through @deepseek-ai/dsh-mcp-client. Pass the ready-made overlay in dsh/computer-use.cordis.yml:

dsh --profile <name> --patch /path/to/dsh-computer-use-mcp/dsh/computer-use.cordis.yml

The model sees the tool as mcp__computer-use__computer_use. See dsh/README.md for the smoke tests that prove the integration keyless.

Claude Code

claude mcp add computer-use -- npx -y dsh-computer-use-mcp

Cursor / generic mcpServers JSON

{
  "mcpServers": {
    "computer-use": { "command": "npx", "args": ["-y", "dsh-computer-use-mcp"] }
  }
}

The computer_use tool

Single tool, dispatched by action. Full JSON schema ships on tools/list.

Typical loop: snapshot → reason over the numbered tree → act by ref (click {ref}) → the post-action feedback re-snapshots and reports the UI diff.

Security model — read before enabling

• MCP has no approval concept: a tool call executes immediately. Enable your host's tool-approval / confirmation layer; do not grant blanket autonomy to untrusted prompts.

• danger-guard statically intercepts dangerous script payloads inside the implementation.

• Per-app "always allow" grants persist under CU_HOME (default ~/.tianshu-computer-use/), separate from any host's own permission store.

• macOS: grant the terminal (or host app) running this server both Accessibility and Screen Recording permissions. The check_permissions action self-probes and reports what is missing.

Environment variables

Development

npm install
npm run build                      # tsup → dist/index.js
npm run smoke                      # SDK-client stdio smoke (tools/list + wait)
node integration/verify-server.mjs # zero-dependency JSON-RPC smoke

integration/dsh-smoke.ts is a keyless integration smoke against DeepSeek Harness's real plugin composition — run it from a deepseek-harness checkout (instructions).

Extracted from huiliyi37/Tianshu-harness — the tianshu (天枢) harness repository. The implementation under impl/ is a frozen snapshot of its computer-use module; this repository is now that code's source of record.

License

GNU AGPL-3.0 — source-available with strong copyleft: modifications distributed or served over a network must be shared. Local use from any MCP host is unrestricted. Commercial licensing available on request.