linux-computer-use-skill

Install From ClawHub

Published on ClawHub as computer-use-linux.

clawhub install computer-use-linux

Related MCP server: mcp-vnc

Positioning

This repository is:

• a top-level skill

• a standalone Linux desktop-control runtime

• a computer-use MCP server for agent ecosystems

It is packaged skill-first instead of depending on any local Claude install.

Why This Exists

The requirement is stricter than "wrap an existing install":

• no dependency on a local Claude app

• no private .node binaries

• no extracted hidden assets

• install the skill, build the server, and use it

This project follows that rule on Linux.

What You Get

• top-level Linux computer-use skill

• standalone MCP server for screenshots, mouse, keyboard, app launch, window/display mapping, and clipboard

• public dependency chain only: Node.js + Python + pyautogui + mss + Pillow + psutil + python-xlib

• first-run runtime bootstrap: the server creates its own virtualenv and installs dependencies automatically

• bundled skill install that copies the full project into ~/.codex/skills/computer-use-linux/project

• extracted TypeScript tool layer wired to a Linux-native Python backend

Status

Implemented in this repository:

• Linux Python helper and runtime bootstrap

• display enumeration and screenshot pipeline

• mouse, keyboard, drag, scroll, and clipboard primitives

• frontmost app, app-under-point, running app, installed app, and window-display lookup paths

• Linux-first skill packaging and bundled project payload

• TypeScript build passing

Still recommended before production use:

• validate on a real Linux host

• test multiple desktop environments and monitor layouts

• test focus, clipboard, and permission edge cases

This session did not have a live Linux machine attached, so runtime behavior on Linux has been implemented and built, but not end-to-end verified on a real Linux desktop.

What Was Fixed In 0.1.1

Version 0.1.1 fixes a Linux packaging regression in the shared system-key blocklist and platform typing. The migrated shared files had been edited into an invalid copied branch, which meant Linux builds were not using a clearly defined Linux shortcut denylist.

This release restores explicit Linux handling for system-level shortcut checks and syncs the fix into both the source tree and the bundled skill payload.

Important Scope

Current desktop-control support is aimed at X11 sessions.

Notable implications:

• X11 desktop sessions are the primary target

• Wayland may block or limit screenshots, focus inspection, clipboard access, and synthetic input depending on compositor policy

• distro / desktop-environment differences can affect behavior

Architecture

flowchart LR
    A[Agent / MCP Client] --> B[linux-computer-use-skill]
    B --> C[Extracted TypeScript MCP tools]
    B --> D[Standalone Python bridge]
    D --> E[pyautogui]
    D --> F[mss + Pillow]
    D --> G[psutil + python-xlib]
    E --> H[Mouse / Keyboard]
    F --> I[Screenshots]
    G --> J[Apps / Windows / Displays / Clipboard]

Install

1. Clone and install Node deps

git clone https://github.com/wimi321/linux-computer-use-skill.git
cd linux-computer-use-skill
npm install
npm run build

2. Start the server

node dist/cli.js

On first launch, the project will automatically:

• create .runtime/venv

• bootstrap pip if needed

• install the Python runtime dependencies from runtime/requirements.txt

MCP Configuration

{
  "mcpServers": {
    "computer-use": {
      "command": "node",
      "args": [
        "/absolute/path/to/linux-computer-use-skill/dist/cli.js"
      ],
      "env": {
        "CLAUDE_COMPUTER_USE_DEBUG": "0",
        "CLAUDE_COMPUTER_USE_COORDINATE_MODE": "pixels"
      }
    }
  }
}

See examples/mcp-config.json.

Skill Install

This repo ships a top-level skill at skill/computer-use-linux.

Option A: Install from ClawHub

clawhub install computer-use-linux

bash skill/computer-use-linux/scripts/install.sh

After installation, the bundled project lives at:

~/.codex/skills/computer-use-linux/project

If CODEX_HOME is set, use that location instead.

Validation Matrix

Validated in this session:

• npm run check

• npm run build

• Python syntax compile check for runtime/linux_helper.py

• bundled skill source integrity checks

• bundled project version sync checks

• review of Linux-specific runtime paths for X11 display discovery, screenshots, clipboard, frontmost app, app enumeration, and window/display lookup

Not yet validated in this session:

• real Linux GUI control

• live screenshot capture on Linux

• foreground-window enforcement against real Linux apps

• Wayland behavior under different compositors

• mixed desktop-environment and multi-monitor edge cases

Runtime Notes

Permissions

Linux desktop control can still be limited by:

• Wayland compositor restrictions

• sandboxed app isolation

• session / remote desktop boundaries

• desktop-environment specific focus and clipboard behavior

Screenshot Filtering

This standalone runtime reports screenshotFiltering: none.

That means screenshot filtering is not compositor-native; gating still happens at the MCP layer.

Platform Scope

This repository is intentionally Linux-only.

Covered capabilities:

• screenshots

• mouse control

• keyboard input

• frontmost app inspection

• installed / running app discovery

• window-to-display mapping

• clipboard access

• app launch

Example Commands

npm run build
node dist/cli.js

node --input-type=module -e "import { callPythonHelper } from './dist/computer-use/pythonBridge.js'; console.log(await callPythonHelper('list_displays', {}));"

Repository Layout

src/
  computer-use/
    executor.ts
    hostAdapter.ts
    pythonBridge.ts
  vendor/computer-use-mcp/
runtime/
  linux_helper.py
  requirements.txt
skill/
  computer-use-linux/
examples/
assets/

Environment Flags

• CLAUDE_COMPUTER_USE_DEBUG=1

• CLAUDE_COMPUTER_USE_COORDINATE_MODE=pixels

• CLAUDE_COMPUTER_USE_CLIPBOARD_PASTE=1

• CLAUDE_COMPUTER_USE_MOUSE_ANIMATION=1

• CLAUDE_COMPUTER_USE_HIDE_BEFORE_ACTION=0

Roadmap

• validate and harden on real Linux hardware

• improve app identity and icon extraction on Linux

• add automated Linux integration tests

• document Wayland-specific limitations and alternatives

License

MIT

Credits

This project preserves and adapts reusable TypeScript computer-use logic recovered from the Claude Code workflow, then replaces the missing private runtime with a fully standalone public Linux implementation.