Skip to main content
Glama
dsh-computer-use-mcp

dsh-computer-use-mcp

by huiliyi37

dsh-computer-use-mcp

English | 中文

Standalone Model Context Protocol server that exposes desktop GUI automation — "computer use" — for macOS and Windows. Any MCP host can drive it: DeepSeek Harness (dsh), Claude Code, Cursor, and every other MCP client.

One tool, computer_use, with 23 actions: accessibility-tree snapshots with numbered refs, click / drag / type (CJK via clipboard) / hotkeys, app launch & focus, menu selection, screenshot artifacts, and a Chrome DevTools Protocol fast path for browser tabs.

Highlights

  • Accessibility-first — snapshots return a numbered tree; every subsequent action addresses elements by ref, not by brittle coordinates.

  • Cross-platform — macOS (AX API via AppleScript/JXA) and Windows (UIA3 COM fast path with managed fallback).

  • Browser CDP fast path — Chrome-family targets automatically switch to a DevTools session for sub-second snapshots and occlusion-proof input (navigate / read_page / js_eval / tabs).

  • Screenshots as image artifacts — vision models receive PNGs; files persist under CU_HOME for audit.

  • Safety rails built in — a static danger-guard intercepts dangerous script payloads; per-app "always allow" grants persist in an owned table.

Install

Node ≥ 20.

npm install -g dsh-computer-use-mcp   # then run: dsh-computer-use-mcp
# or one-shot:
npx -y dsh-computer-use-mcp

Hook it to your host

DeepSeek Harness (dsh)

dsh bridges MCP servers through @deepseek-ai/dsh-mcp-client. Pass the ready-made overlay in dsh/computer-use.cordis.yml:

dsh --profile <name> --patch /path/to/dsh-computer-use-mcp/dsh/computer-use.cordis.yml

The model sees the tool as mcp__computer-use__computer_use. See dsh/README.md for the smoke tests that prove the integration keyless.

Claude Code

claude mcp add computer-use -- npx -y dsh-computer-use-mcp

Cursor / generic mcpServers JSON

{
  "mcpServers": {
    "computer-use": { "command": "npx", "args": ["-y", "dsh-computer-use-mcp"] }
  }
}

The computer_use tool

Single tool, dispatched by action. Full JSON schema ships on tools/list.

Group

Actions

Read / probe

check_permissions, list_apps, snapshot, find, wait_for, wait

Input

click, double_click, right_click, scroll, drag, type, set_value, key, paste_text

App / window

focus_app, launch_app, menu_select

Browser (CDP)

navigate, read_page, js_eval, tabs, browser_adopt

Typical loop: snapshot → reason over the numbered tree → act by ref (click {ref}) → the post-action feedback re-snapshots and reports the UI diff.

Security model — read before enabling

  • MCP has no approval concept: a tool call executes immediately. Enable your host's tool-approval / confirmation layer; do not grant blanket autonomy to untrusted prompts.

  • danger-guard statically intercepts dangerous script payloads inside the implementation.

  • Per-app "always allow" grants persist under CU_HOME (default ~/.tianshu-computer-use/), separate from any host's own permission store.

  • macOS: grant the terminal (or host app) running this server both Accessibility and Screen Recording permissions. The check_permissions action self-probes and reports what is missing.

Environment variables

Variable

Effect

CU_HOME

Data directory for grants and screenshot artifacts (default ~/.tianshu-computer-use)

CU_SESSION_ID

Artifact subdirectory name per session (default mcp)

RIVET_CU_CDP=0

Disable the browser CDP fast path

RIVET_CU_FEEDBACK=0

Disable post-action UI feedback summaries

RIVET_CU_HOST=0

Disable the script host (macOS JXA / Windows PowerShell channel)

RIVET_CU_COM=0

Disable the Windows UIA3 COM fast path (falls back to the managed builder)

Development

npm install
npm run build                      # tsup → dist/index.js
npm run smoke                      # SDK-client stdio smoke (tools/list + wait)
node integration/verify-server.mjs # zero-dependency JSON-RPC smoke

integration/dsh-smoke.ts is a keyless integration smoke against DeepSeek Harness's real plugin composition — run it from a deepseek-harness checkout (instructions).

Extracted from huiliyi37/Tianshu-harness — the tianshu (天枢) harness repository. The implementation under impl/ is a frozen snapshot of its computer-use module; this repository is now that code's source of record.

License

GNU AGPL-3.0 — source-available with strong copyleft: modifications distributed or served over a network must be shared. Local use from any MCP host is unrestricted. Commercial licensing available on request.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors