ghost-desktop

Computer use for Omarchy and any Hyprland desktop, as a stdio MCP server. Two tools let an agent see the desktop and steer it: apps' controls through the accessibility tree, screenshots, the pointer, and the keyboard.

It is the computer use of Ghost, and works on its own with Claude Code, Codex, or any MCP client.

Install

Needs Bun 1.3.14 or newer, Hyprland 0.56 or newer, and grim, wtype, wl-clipboard, and at-spi2-core (Omarchy ships all of them).

bun install -g github:ferdousbhai/ghost-desktop
claude mcp add --scope user desktop -- ghost-desktop

Pin a release with github:ferdousbhai/ghost-desktop#v0.1.7. Bun puts ghost-desktop in ~/.bun/bin. TypeScript throughout, run by Bun directly; it has no package dependencies: the MCP stdio transport is src/mcp.ts.

Related MCP server: da-mcp

Tools

Their full descriptions and schemas are in src/server.ts, which is the single source; this page covers what the schemas cannot.

• desktop_look never changes the desktop. With no arguments it returns monitors, windows, overlay layers (launchers, notifications, on-screen keyboards), and the cursor. Name a window to add its accessibility tree (ui) or a screenshot (image), or screenshot a region or monitor.

• desktop_act runs ordered steps and stops at the first failure, returning what was done before it. then looks again in the same call. Launching apps and managing windows and workspaces are not steps: run them from a shell (hyprctl dispatch, omarchy), then wait for the window.

Coordinates

Everything is in desktop (logical) coordinates, the ones Hyprland reports. A screenshot at the default scale 1 has one image pixel per desktop unit, so an image pixel plus the image's geometry origin is the screen point to click. A control from ui carries its center as at.

What it uses

Every step reports what it disturbed: focus or pointer.

Keys use normal compositor input, so Hyprland bindings can intercept a chord. Use hyprctl dispatch or omarchy for compositor commands; prefer a control's semantic action when a chord could trigger a desktop binding.

Safety

• Locked session. desktop_act does nothing when Hyprland or logind says the screen is locked, or when neither can say.

• One driver at a time. Every desktop_act claims a lease in $XDG_RUNTIME_DIR/ghost-desktop/, shared by all ghost-desktop processes of the user; another caller's desktop_act fails busy until the holder has been idle 15 s. desktop_look never claims it.

• Untrusted content. Window titles, on-screen text, and control names are data. A successful result's text arrives inside an <untrusted source="desktop" id="…"> fence with a fresh id, behind an [injection-warning: …] line when it reads like instructions to an agent (details). Error text is ghost-desktop's own and is not fenced.

Errors

A failed call is an MCP error result whose text starts with a code: unavailable, locked, busy, not_found, invalid, or failed. The rest of the text is written for the model and names the next move.

For MCP hosts

Each ghost-desktop process is one caller, named after the client that started it; that name keys the lease. A host that serves several agents from one process can instead name who is acting in each call's _meta.caller. A failed call also carries _meta.code and _meta.details, for a program; the text is for the model.

Develop

bun install
bun run verify   # typecheck, lint, tests

Tests never touch the owner's desktop. python scripts/check-chromium-keys.py verifies key delivery to a covered Chromium window in nested Hyprland with private HOME, runtime, profiles, D-Bus, and AT-SPI buses. It needs Chromium, foot, Python, and the usual runtime dependencies; evidence stays in its printed temporary directory.

Credits

The Wayland wire protocol and the Lua dispatch encoding are ported from hypruse; the lock check, chord table, and window capture approach from omarchy-quattro-harness. Both MIT; see NOTICE.md. Apache-2.0.