Skip to main content
Glama
deenesjakoruzh
YV17labs

GhostDesk

by YV17labs
OverviewSchemaRelated ServersScoreDiscussions
Rust
Remote

One binary, one MCP endpoint, three desktops. The tool surface is the same everywhere — an agent calls mouse_click and app_launch without knowing which desktop it is on.

Linux

macOS

Windows

How it runs

the Docker container, in one command

a native binary

a native binary

The desktop it drives

a virtual one, shipped inside the image

the Mac in front of you

the PC in front of you

Sandboxed

yes — disposable, one per agent

no

no

Driven through

zwlr_virtual_pointer_v1, Sway IPC, grim

Quartz Event Services, Accessibility, screencapture

SendInput, GDI, EnumWindows

Linux is the server. The container is the deployment this README shows unless it says otherwise, and the only one of the three with a sandbox around it. macOS and Windows run the very same server against a real desk — no container, and therefore no isolation. Jump to macOS or Windows, or read what differs between the three.

Quick start

1. Run the container

One command, plain HTTP, no password. Fine for kicking the tires on a laptop you trust — not fit for anything beyond that. Ready to harden it? Jump to Secure local run.

docker run -d --name ghostdesk-demo \
  --shm-size 2g \
  -p 3000:3000 \
  -p 6080:6080 \
  ghcr.io/yv17labs/ghostdesk:latest

The latest image ships with Firefox, the foot terminal, mousepad (text editor), galculator, and passwordless sudo for the agent user — enough to demo a browsing + note-taking workflow out of the box. Need a different app set? Build your own on top of base — see Custom image.

The container boots in the dev posture: plain HTTP on both ports, every auth gate disarmed on purpose. You'll see warnings in the logs reminding you of that — they go away once you follow the secured path below.

2. Connect your AI

GhostDesk speaks MCP over the Streamable HTTP transport — any MCP-compatible client can drive it. Point your client at http://localhost:3000/mcp:

Claude Desktop / Claude Code

{
  "mcpServers": {
    "ghostdesk": {
      "type": "http",
      "url": "http://localhost:3000/mcp"
    }
  }
}

SpecterChat — the chat client we build for this, open source: YV17labs/SpecterChat. Most chat UIs drop the image an MCP tool returns — they render it or they forward it to the model, rarely both — and a screen_shot() the model never sees is the whole product missing. SpecterChat displays it inline and sends it back as base64. It talks to any OpenAI-compatible endpoint (llama.cpp, vLLM, Ollama, LM Studio), so it pairs with the local stacks below; macOS, Linux and Windows builds are on its releases page.

Any other MCP-compatible client — same URL, no headers, no auth. That's the whole demo posture.

3. Watch your agent work

Open http://localhost:6080/ in your browser to see the virtual desktop in real time. No password prompt — the dev posture skips it.

Service

URL

MCP server

http://localhost:3000/mcp

noVNC (browser)

http://localhost:6080/

Health probes

http://localhost:3000/health/{live,ready,startup}

The probes are what the container's HEALTHCHECK reads, and they answer about the desktop rather than about the processes: ready goes down when the compositor stops answering the window seam, live when the connection behind the virtual pointer and keyboard is gone — a state in which every tool still replies and none of them does anything.

Give your agent a first prompt to confirm the wiring is right:

"Take a screenshot of the desktop, list the installed applications, then open Firefox and go to wikipedia.org."

You should see Firefox launch in the noVNC tab, the URL bar fill in, and the page load — all under your agent's control.

4. When you're done

docker stop ghostdesk-demo && docker rm ghostdesk-demo

The demo run creates no named volume, so this leaves nothing behind.

Related MCP server: umbriel

Tools

Fourteen tools, named verb_noun, and this is the whole surface — no hidden endpoint, no second protocol. Defaults are in parentheses, ? marks an optional parameter, and every coordinate is a pixel offset in the last screen_shot(): the moment the screen changes, coordinates computed from the previous capture are stale.

Screen

Tool

Parameters

Returns

screen_shot

region? — {x, y, width, height}, cropped at native resolution · format "webp" | "png" (webp) · stabilize bool (true) — wait up to 5 s for the screen to settle · quality 1–100 (50, WebP only; raise it for fine fonts or design surfaces)

one image block — {"type": "image", "data": "<base64>", "mimeType": "image/webp"}

Mouse and keyboard

The seven input tools answer with the same verdict, and screen_changed is the field worth branching on: false means the act landed on nothing. It is a signal, not an error — the answer is a fresh capture, never a retry at the same coordinates.

{"action": "Clicked left at (612, 335)", "screen_changed": true, "reaction_time_ms": 180}

Tool

Parameters

mouse_move

x int · y int

mouse_click

x · y · button "left" | "middle" | "right" (left)

mouse_double_click

x · y · button (left)

mouse_drag

from_x · from_y · to_x · to_y · button (left)

mouse_scroll

x · y · direction "up" | "down" | "left" | "right" (down) · amount 1–5 wheel notches (3)

key_type

text string — Unicode, newlines and tabs, layout-independent

key_press

keys string — one chord, + between tokens. Modifiers: ctrl/control, alt/option, shift, super/meta/win/cmd/command. Named keys: return/enter, escape/esc, backspace, delete, tab, space, home/end, pageup/pagedown, left/right/up/down, f1–f12

Past a sentence or two, clipboard_set(text) plus the paste chord beats key_type: it is instant, and immune to autocomplete and to the app's own key handlers.

Clipboard

Tool

Parameters

Returns

clipboard_get

—

the clipboard as text — an empty string when it is empty or holds something that is not text

clipboard_set

text string

Clipboard set (N characters)

Apps

Tool

Parameters

Returns

app_list

—

result[], one entry per installed app: name, exec. This is the launch whitelist, and exec is the exact string app_launch takes

app_running

—

result[], one entry per real client window: app, title, pid, focused

app_launch

command string — an exec from app_list, arguments not accepted · wait_for_window bool (true)

pid, log_file, action, plus window and window_wait_ms once a window appeared — and the settled screen as an image block, so no follow-up screen_shot() is needed

app_status

pid int — one returned by app_launch · lines int (50)

pid, running, log_file, tail — the tail of the captured stdout/stderr

app_list is a whitelist rather than a hint: app_launch refuses anything outside it, arguments included, whatever name the model sends.

On the wire

A call is ordinary MCP over Streamable HTTP — POST /mcp, whose Accept header has to name both application/json and text/event-stream or the endpoint answers 406.

// the params of a tools/call request
{"name": "mouse_click", "arguments": {"x": 612, "y": 335}}

The result carries its payload twice — once in structuredContent, once as a text block holding the same JSON, which is what a client older than structured output reads. screen_shot is the exception and answers with an image block.

Two headers are GhostDesk's own: Authorization: Bearer …, required once a cert is mounted (Secure local run), and GhostDesk-Model-Space, for models that emit normalised coordinates (Model requirements).

Model requirements

Your inference stack must cover four capabilities — all four are mandatory:

  1. Text + vision — the agent perceives the desktop through screenshots and needs a model that can interpret them.

  2. Tool use — GhostDesk exposes its tools as function calls; the model must be able to invoke them.

  3. MCP client — the host needs to speak Streamable HTTP MCP to reach the GhostDesk server.

  4. WebP image support — GhostDesk returns screenshots as WebP by default to keep payloads small and inference fast. A stack that can only decode PNG or JPEG will not work out of the box.

Points 3 and 4 are where most stacks fall short, and both halves have an answer here: SpecterChat on the client side, and the llama.cpp forks below on the inference side.

Coordinate space — GhostDesk-Model-Space header

By default no header is needed: Claude and the other major frontier LLMs work out of the box. Qwen3.x need the client to send GhostDesk-Model-Space: 1000 on every MCP request.

Example MCP client config:

{
  "mcpServers": {
    "ghostdesk": {
      "url": "http://localhost:3000/mcp",
      "headers": {
        "GhostDesk-Model-Space": "1000"
      }
    }
  }
}

Running locally

Three backends are tested here. Two are llama.cpp forks we maintain, both kept current with upstream, both adding the WebP decoding upstream still lacks — the day it lands there, they are archived and this points at upstream directly. The third is upstream mlx-vlm, on Apple Silicon.

  • YV17labs/llama-cpp-webp — branch feature/webp. Start here. WebP decoding and nothing else on top of upstream, so it stays close to master and inherits its backend work. It is the faster of the two on Metal and on CUDA — on an Apple Silicon Mac or an NVIDIA card, this is the one to run.

  • YV17labs/llama-cpp-turboquant-webp — branch feature/turboquant-webp. The same WebP support plus the turbo-quant KV cache (--cache-type-v turbo3). Still maintained and still tracking upstream, but turbo quant is no longer where the interest is, and this is no longer the first recommendation.

  • Blaizzy/mlx-vlm — upstream as it ships, since it decodes WebP already and needs nothing from us. Apple Silicon only, and the MLX path rather than a llama.cpp one.

Run whatever local model you like — nothing in GhostDesk is pinned to one. The one behind my own runs is Qwen3.6-35B-A3B: 35B parameters with only 3B active per token, and on desktop control that ratio is the whole point — the agent decides where to click on every step, so tokens per second is what you feel.

The commands

One tested invocation per backend. They do not take the same flags, so each gets its own rather than one command with a switch — and the model in them is an example, not a requirement: swap in whatever you run.

llama-cpp-webp — --image-min-tokens 1024 is the one that matters for desktop control: it floors how much of the token budget a screenshot gets, and a screenshot the model reads at too coarse a scale is where off-target clicks come from.

build/bin/llama-server \
  --model ~/Models/Qwen3.6-35B-A3B-Q4_K_M.gguf \
  --mmproj ~/Models/Qwen3.6-35B-A3B-mmproj-F16.gguf \
  --alias 'Qwen3.6-35B-A3B-Q4_K_M' \
  --host 127.0.0.1 --port 8080 \
  --ctx-size 131072 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --flash-attn on \
  --image-min-tokens 1024 \
  --reasoning on --reasoning-format deepseek --reasoning-preserve \
  --jinja

llama-cpp-turboquant-webp — the KV cache goes to --cache-type-v turbo3, and --cache-reuse 256 keeps the prefix across turns, which a desktop session hits constantly: the conversation grows by one screenshot and one tool result at a time. --spec-type draft-mtp turns on the model's own multi-token-prediction draft head, so speculative decoding needs no second model loaded beside it.

build/bin/llama-server \
  --model ~/Models/Qwen3.6-35B-A3B-Q4_K_M.gguf \
  --mmproj ~/Models/Qwen3.6-35B-A3B-mmproj-F16.gguf \
  --alias 'Qwen3.6-35B-A3B-Q4_K_M' \
  --host 127.0.0.1 --port 8080 \
  --ctx-size 131072 \
  --cache-type-k q8_0 --cache-type-v turbo3 \
  --flash-attn on \
  --spec-type draft-mtp --spec-draft-n-max 3 \
  --reasoning on --reasoning-format deepseek \
  --jinja --cache-reuse 256

mlx-vlm — the flags are the same three ideas under other names: --kv-bits 8 --kv-quant-scheme uniform quantises the KV cache, --enable-thinking turns reasoning on, and --draft-kind mtp is the multi-token-prediction draft head. That last one is the difference that costs something: here the head is a second set of weights on disk (--draft-model), not a switch on the model already loaded.

.venv/bin/mlx_vlm.server \
  --model ~/Models/Qwen3.6-35B-A3B-MLX-4bit \
  --host 127.0.0.1 --port 8080 \
  --kv-bits 8 --kv-quant-scheme uniform \
  --enable-thinking \
  --draft-model ~/Models/Qwen3.6-35B-A3B-MLX-mtp --draft-kind mtp

Each of the three exposes an OpenAI-compatible endpoint on http://127.0.0.1:8080; point your MCP host's inference backend at it — SpecterChat's endpoint field takes that URL as is — and remember the GhostDesk-Model-Space: 1000 header for the Qwen family.

Why GhostDesk?

Browser automation tools (Playwright, Puppeteer, Selenium…) were built for human test engineers driving a browser with selectors. They do one thing, and they do it well — inside the browser.

GhostDesk is built from the other end: for AI agents, driving everything a desktop runs. Browsers, native apps, IDEs, terminals, office suites, legacy software, internal tools. If it renders pixels on screen, your agent can see it and use it — in one conversation, across many applications, without a line of glue code.

You don't write selectors. You write a prompt:

"Open the CRM, export last month's leads as CSV, open LibreOffice Calc, build a pivot table, screenshot the chart, and email it to the team."

The agent opens the browser, logs in, downloads the file, switches to LibreOffice, processes the data, captures the result, composes the email, sends it. One prompt, multiple apps, fully autonomous — no glue code, no per-site scraper, no brittle selector chain.

That is what agents using a desktop looks like.

Runs on models you can actually host

Desktop control needs to be fast — an agent that takes twelve seconds to decide where to click is unusable. The local path is first-class here, and it is three concrete things rather than a promise: screenshots ship as WebP so a capture costs a small payload, the coordinate space a model emits is one header away, and the two llama.cpp forks below carry the WebP decoding upstream still lacks. No API bill, and no screenshot of your desktop leaving your network.

Frontier models (Claude, GPT-4o, Gemini) work too and remain the smoothest path — but they are not the bar. See Model requirements for the supported stacks and the one coordinate-space setting that matters.

How it works

GhostDesk drives a desktop and exposes it as an MCP server. The three ways to run it are the three columns at the top of this page — the container, a macOS binary, a Windows binary — and the tool surface is identical in all of them. The container is what the rest of this README shows unless it says otherwise.

The agent perceives the screen by calling screen_shot(), which captures the full desktop at native resolution and returns it as WebP (or PNG). An optional region= argument can crop to a sub-rectangle when the agent explicitly wants to narrow its focus.

This works with any application — web apps, native apps, legacy software, Canvas, WebGL.

Built in Rust

GhostDesk is a single compiled binary. It links libc and nothing else — no interpreter, no virtual environment, no package tree to harden at build time.

The tool host mounts itself on the HTTP transport, each domain is a service resolved by type, and the whole dependency graph is verified at boot — a missing binding is a startup error, never a runtime surprise. The endpoint is closed by default: a guard has to bind before /mcp answers anything at all.

The operating system sits behind five traits — input, screen, windows, clipboard, application catalogue — and each OS is one directory implementing those five, under crates/platform/src. Nothing above that boundary names a desktop, which is what made the second one possible at all, and the third one routine.

On Linux the compositor is driven from pure Rust: GhostDesk speaks zwlr_virtual_pointer_v1 and zwp_virtual_keyboard_v1 directly over the Wayland socket, with an XKB keymap it generates on the fly — which is why text entry produces identical output on a French AZERTY host and a US QWERTY one. On macOS the same five contracts are answered by Quartz Event Services, the Accessibility API, screencapture and the pasteboard. On Windows they are answered by SendInput, EnumWindows, GDI, the Win32 clipboard and the Start Menu — all in process, because Windows is the one of the three that ships no capture or clipboard tool to shell out to.

Secure local run (TLS + auth)

The Quick start above drops every gate so you can kick the tires in thirty seconds. The moment you want to expose this to anything beyond your own laptop — another machine on your LAN, a devcontainer port-forward on an untrusted network, a teammate's browser — flip to the secured posture: real TLS + bearer-token auth on MCP + password prompt on noVNC.

GhostDesk couples TLS and auth: mount a cert and you get wss:// + bearer-token on MCP + a single-password prompt on noVNC (see Security → Auth ≡ TLS). mkcert issues a browser-trusted cert for localhost in two commands:

# Issue a locally-trusted cert (first time only — installs a local CA in your trust store)
mkcert -install
mkdir -p tls
mkcert -cert-file tls/server.crt -key-file tls/server.key localhost 127.0.0.1 ::1

# Generate the MCP and VNC secrets
export GHOSTDESK_AUTH__TOKEN=$(openssl rand -hex 32)
export GHOSTDESK_VNC_PASSWORD=$(openssl rand -hex 16)

Pick a container name that matches the agent's role — sales-agent, research-agent, accounting-agent… Below we use my-agent as a placeholder; replace it everywhere in the command.

# Run the container — cert mounted, TLS + auth enabled everywhere
docker run -d --name ghostdesk-my-agent \
  --restart unless-stopped \
  --cap-add SYS_ADMIN \
  --shm-size 2g \
  -p 3000:3000 \
  -p 6080:6080 \
  -v ghostdesk-my-agent-home:/home/agent \
  -v "$PWD/tls/server.crt:/etc/ghostdesk/tls/server.crt:ro" \
  -v "$PWD/tls/server.key:/etc/ghostdesk/tls/server.key:ro" \
  -e GHOSTDESK_AUTH__TOKEN \
  -e GHOSTDESK_VNC_PASSWORD \
  -e TZ=America/New_York \
  -e LANG=en_US.UTF-8 \
  ghcr.io/yv17labs/ghostdesk:latest

echo "MCP token:    $GHOSTDESK_AUTH__TOKEN"
echo "VNC password: $GHOSTDESK_VNC_PASSWORD"

Once the container is up, update your MCP client config — same shape as the demo, now over https:// with a bearer token:

Claude Desktop / Claude Code

{
  "mcpServers": {
    "ghostdesk": {
      "type": "http",
      "url": "https://localhost:3000/mcp",
      "headers": {
        "Authorization": "Bearer <paste $GHOSTDESK_AUTH__TOKEN here>"
      }
    }
  }
}

Any other MCP-compatible client — same URL, plus an Authorization: Bearer <token> header in whatever form your client accepts.

Then open https://localhost:6080/ in your browser — the mkcert CA installed by mkcert -install is already in your trust store, so the browser accepts the cert with no warning. noVNC will prompt for $GHOSTDESK_VNC_PASSWORD.

Going to production? Swap the mkcert leaf for a real cert, source both secrets from your secret manager, and front port 6080 with an identity-aware proxy — SECURITY.md has the full contract.

--cap-add SYS_ADMIN — Required by Electron apps (VS Code, Slack, etc.) and other applications that need Linux user namespaces to run their sandbox. Safe to remove if you don't need them.

The named volume persists the agent's home directory across restarts — browser passwords, bookmarks, cookies, downloads, and desktop preferences are all preserved. On the first run, Docker automatically seeds the volume with the default configuration from the image.

Running many agents

One agent is one container. Two of them share nothing — not the filesystem, not the desktop, not the clipboard — so a second agent is a second port pair, a second volume and a second name. What differs between two of them is the system prompt you give the model, the applications in the image (Custom image), and the networks you attach the container to.

Three agents, one compose file

# docker-compose.yml — 3 specialized agents, one command
#
# Prerequisites: the TLS cert + key at ./tls and the two secrets
# (GHOSTDESK_AUTH__TOKEN, GHOSTDESK_VNC_PASSWORD) in your environment or a
# .env file. Generate both exactly as shown in the Secure local run
# section above. See SECURITY.md for the production secret-handling
# contract.

x-ghostdesk-defaults: &ghostdesk-defaults
  image: ghcr.io/yv17labs/ghostdesk:latest
  restart: unless-stopped
  cap_add: [SYS_ADMIN]
  shm_size: 2g
  environment:
    - GHOSTDESK_AUTH__TOKEN
    - GHOSTDESK_VNC_PASSWORD
    - TZ=America/New_York
    - LANG=en_US.UTF-8

services:
  sales-agent:
    <<: *ghostdesk-defaults
    container_name: ghostdesk-sales-agent
    ports: ["3001:3000", "6081:6080"]
    volumes:
      - ghostdesk-sales-agent-home:/home/agent
      - ./tls/server.crt:/etc/ghostdesk/tls/server.crt:ro
      - ./tls/server.key:/etc/ghostdesk/tls/server.key:ro

  research-agent:
    <<: *ghostdesk-defaults
    container_name: ghostdesk-research-agent
    ports: ["3002:3000", "6082:6080"]
    volumes:
      - ghostdesk-research-agent-home:/home/agent
      - ./tls/server.crt:/etc/ghostdesk/tls/server.crt:ro
      - ./tls/server.key:/etc/ghostdesk/tls/server.key:ro

  accounting-agent:
    <<: *ghostdesk-defaults
    container_name: ghostdesk-accounting-agent
    ports: ["3003:3000", "6083:6080"]
    volumes:
      - ghostdesk-accounting-agent-home:/home/agent
      - ./tls/server.crt:/etc/ghostdesk/tls/server.crt:ro
      - ./tls/server.key:/etc/ghostdesk/tls/server.key:ro

volumes:
  ghostdesk-sales-agent-home:
  ghostdesk-research-agent-home:
  ghostdesk-accounting-agent-home:
docker compose up -d

Each service is the same image with its name, its ports and its volume changed. Per agent, that costs:

Per agent

What it takes

Two published ports

3000 for MCP, 6080 for noVNC — one pair per container, mapped to whatever the host has free

One named volume

the agent's /home/agent: browser profile, cookies, downloads, desktop settings, kept across restarts

One shm_size: 2g

shared memory for the browser and the other GPU-accelerated apps. It is a cap rather than a reservation — pages are allocated as they are touched — but every container may claim up to that much of the host's RAM

One desktop

Sway, mako, wayvnc, websockify and the MCP server, under one supervisord

Nothing coordinates the instances: no scheduler, no shared state, no leader. Ten agents are ten docker runs, and stopping one is docker rm.

Container isolation

The container boundary is the only isolation GhostDesk has, and it is Docker's rather than the server's: separate filesystem, process and network namespaces, one volume per agent, and a docker rm that takes the desktop and everything the agent did to it. The MCP port and the noVNC port are the two doors through that boundary, which is why Secure local run puts TLS and a credential on both.

Two things it does not give you, and both belong to the deployment. Segmentation between agents is the first: a container reaches whatever the networks you attached it to reach, so an agent that must not see the internet is one you attach only to Docker networks with no route off the host. Per-user identity on either door is the second — the token and the VNC password are one credential each, shared by every caller. SECURITY.md draws the whole line, in scope against out of scope.

Watching one work

Every instance serves its own noVNC, so supervision is one browser tab per agent — https://localhost:6081/ for the sales agent above, 6082 for research, 6083 for accounting (the compose file mounts a cert, so those are the secured posture's URLs). The tab is not read-only: take the mouse and keyboard whenever you want, and the agent's next screen_shot() sees whatever you left on screen.

Custom image

The base tag provides GhostDesk without any pre-installed GUI application — just the virtual desktop, VNC, and the MCP server. Use it to build your own image with only the tools you need:

FROM ghcr.io/yv17labs/ghostdesk:base

RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        chromium-browser \
        libreoffice-calc \
    && rm -rf /var/lib/apt/lists/*
docker build -t my-agent .

See the project's Dockerfile for a complete example.

Tag

Description

latest, X.Y.Z, X.Y

Full image — Firefox, foot terminal, mousepad, galculator, passwordless sudo

base, base-X.Y.Z, base-X.Y

Minimal image — no GUI app, meant to be extended

Run on macOS, without the container

The container ships a Linux desktop. GhostDesk is also just a binary, and on macOS that binary drives the Mac in front of you — the same tools, the same MCP endpoint, nothing in between. The five OS seams are answered by Quartz Event Services for the pointer and keyboard, the Accessibility API for windows, screencapture for frames, the pasteboard for the clipboard, and .app bundles for the catalogue.

There is no sandbox on this path. Every isolation guarantee in Container isolation belongs to the container. A native run hands the agent your real mouse, your real keyboard, your real screen and your real applications, with your own permissions. Run it on a machine you are willing to hand over, and watch it.

1. Build and install

From a clone of the repository — Build from source has the toolchain it needs:

cargo install --path apps/ghostdesk --locked

2. Grant the two permissions

macOS gates input and capture behind privacy settings that cannot be requested from code, so you grant them by hand, once, in System Settings ▸ Privacy & Security:

Setting

What it buys

Accessibility

mouse_*, key_*, and every window operation

Screen Recording

screen_shot, and window titles

Miss one and the affected tools fail naming the setting to open, rather than returning a black image or silently doing nothing.

The grant is bound to the binary's path and its signature, which is macOS's rule and not ours: rebuild ghostdesk and the grant is revoked, so both permissions have to be re-granted after every cargo install. Nothing can script this away — TCC exists precisely so that no process can grant itself the thing.

3. Run it

NESTRS_ENV_PREFIX=GHOSTDESK GHOSTDESK_IDLE__TIMEOUT_SECS=0 ghostdesk

Both variables are load-bearing:

  • NESTRS_ENV_PREFIX=GHOSTDESK has to be on the process — see Configuration. Without it every setting is read under its stock NESTRS_* name instead, and none of the GHOSTDESK_* names below reach the server.

  • GHOSTDESK_IDLE__TIMEOUT_SECS=0 disarms the idle sweep. Armed, thirty minutes of MCP silence closes every open window — the right behaviour for a disposable container desktop, and the wrong one for your laptop.

The server binds 127.0.0.1:3000 and serves with no token (posture loopback_open); point your MCP client at http://localhost:3000/mcp exactly as in Connect your AI. There is no noVNC endpoint — the desktop is the one you are looking at.

Run on Windows, without the container

The same binary once more, driving the PC in front of you — the same tools again, the same MCP endpoint. The five OS seams are answered by SendInput for the pointer and keyboard, EnumWindows and WM_CLOSE for windows, GDI for frames, the Win32 clipboard, and Start Menu shortcuts for the catalogue.

Nothing here shells out to a helper process. Linux has grim and wl-clipboard, macOS has screencapture and pbcopy; Windows ships neither, so this is the one host that captures and copies in process — which also means a scaled capture is scaled by the blit, not by decoding and resizing a full-resolution frame afterwards.

There is no sandbox on this path. Every isolation guarantee in Container isolation belongs to the container. A native run hands the agent your real mouse, your real keyboard, your real screen and your real applications, with your own permissions. Run it on a machine you are willing to hand over, and watch it.

1. Build and install

GhostDesk links a WebP encoder written in C, so the build needs a C toolchain. Install the Visual Studio Build Tools with the Desktop development with C++ workload (Visual Studio itself works too), then, from a clone of the repository:

cargo install --path apps/ghostdesk --locked

2. Grant nothing — but run it as yourself

Windows puts none of this behind a privacy setting, so unlike macOS there is nothing to click. What it gates instead is integrity level, and two rules follow from that:

  • Run GhostDesk as the signed-in user, in an interactive session. A Windows service lives in session 0, which has no desktop at all. GhostDesk refuses to boot there, naming the reason, rather than accepting clicks that go nowhere.

  • An unelevated GhostDesk cannot reach an elevated window — Task Manager, an installer, anything started with Run as administrator. Windows discards that input in silence, which is exactly why the boot check exists. Starting GhostDesk elevated lifts the restriction and hands the agent an administrator's desktop; do that deliberately or not at all.

While a UAC prompt or the lock screen is in front, the session belongs to Winlogon and no application can drive it — GhostDesk included. It reports that instead of reporting a healthy desk, so a supervisor sees a server that cannot work rather than one that appears to.

3. Run it

$env:NESTRS_ENV_PREFIX = "GHOSTDESK"
$env:GHOSTDESK_IDLE__TIMEOUT_SECS = "0"
ghostdesk

Both variables are load-bearing, for the same two reasons they are on macOS: NESTRS_ENV_PREFIX=GHOSTDESK is what makes every GHOSTDESK_* name below reach the server, and GHOSTDESK_IDLE__TIMEOUT_SECS=0 disarms the idle sweep, which would otherwise close your own windows after thirty minutes of MCP silence.

The server binds 127.0.0.1:3000 and serves with no token (posture loopback_open); point your MCP client at http://localhost:3000/mcp exactly as in Connect your AI. There is no noVNC endpoint — the desktop is the one you are looking at.

What differs from the container

Container (Linux)

Native (macOS)

Native (Windows)

Desktop

virtual, disposable, sandboxed

yours

yours

Primary modifier

ctrl

cmd

ctrl

App catalogue

.desktop entries

.app bundles in /Applications, /System/Applications, their Utilities, and ~/Applications

Start Menu shortcuts, machine-wide and per-user

Supervision

noVNC on :6080

your own screen

your own screen

Screen geometry

GHOSTDESK_SCREEN__WIDTH / _HEIGHT

the main display, at native pixel size

the primary display, at native pixel size

Permissions

none

Accessibility + Screen Recording, granted by hand

none to grant — integrity level decides what it can reach

Windows are closed by

Sway IPC

the Accessibility close button

WM_CLOSE

The modifier is not something you configure, and it is not cosmetic. The server publishes the desktop and its primary modifier in the tool descriptions, built from the same constant the key table presses, so the model is told cmd+c on macOS and ctrl+c on Linux and Windows. Every modifier name resolves on all three desktops — ctrl, alt, option, super, meta, win, cmd, command — but on macOS ctrl+c presses Control and puts a control character in the field, which is why the instruction is published rather than assumed.

The app catalogue is a whitelist on all three, and it is what an agent may launch: nothing outside it can be started, whatever name the model sends. On Windows that means the Start Menu, and a Store application whose shortcut points at a package rather than at an executable stays out of it — a catalogue that listed what it cannot start would be a whitelist that lies.

Running the binary on a Linux host instead of the container works the same way, with the Wayland stack's own expectations: a Sway session for the window seam, grim for capture, wl-clipboard for the clipboard — see Build from source for the command and the caveat.

Configuration

Every variable GhostDesk reads is namespaced under GHOSTDESK_*. Standard POSIX variables (TZ, LANG) are kept as-is so the existing Unix ecosystem keeps working.

The image sets NESTRS_ENV_PREFIX=GHOSTDESK, and that single variable is what makes every setting below read GHOSTDESK_HTTP__PORT rather than NESTRS_HTTP__PORT. There is no second spelling and no translation layer — one name, one place to look it up.

Read them as GHOSTDESK_<NAMESPACE>__<KEY>: the double underscore separates the namespace from the setting. http and mcp are the server's transport namespaces; screen, idle and auth are GhostDesk's, one per feature module that owns settings. A single underscore (GHOSTDESK_VNC_PASSWORD) marks a container-level knob the entrypoint consumes itself, never reaching the server.

NESTRS_ENV_PREFIX is the one name no prefix can rename, and it has to be on the process before the server starts — a .env file is read after it has already chosen which cascade to read. Both images bake it and the Justfile exports it, so a container run and a nestrs run dev both carry it; a binary you start any other way needs NESTRS_ENV_PREFIX=GHOSTDESK in its environment, or every variable below is read under its stock NESTRS_* name instead.

Secrets (required under TLS — the prod container refuses to boot without them)

Variable

Description

GHOSTDESK_AUTH__TOKEN

Bearer token required on every MCP request. Generate with openssl rand -hex 32.

GHOSTDESK_VNC_PASSWORD

Password for wayvnc. RFB security type 2 carries a password and no username, so the noVNC overlay prompts for this one value. Generate with openssl rand -hex 16.

Both are plain environment variables. Wire them from your secret store (secretKeyRef on Kubernetes, Docker secrets / Vault / AWS SM on compose) — see SECURITY.md for the full contract.

Runtime knobs

Variable

Default

Description

GHOSTDESK_HTTP__PORT

3000

MCP server listening port

GHOSTDESK_HTTP__HOST

127.0.0.1 (standalone) / 0.0.0.0 (container)

Bind address for the MCP endpoint. Defaults to loopback per MCP transports spec; the container's entrypoint exports 0.0.0.0 so Docker's port-publishing layer can reach it.

GHOSTDESK_HTTP__CORS_ORIGINS

(empty)

Comma-separated list of Origin headers accepted from browser clients (e.g. https://app.example.com,https://localhost:8080). Non-browser clients (Claude Desktop, SDKs, curl) send no Origin and are always allowed. Required for any browser-based MCP UI: without it no CORS layer is mounted at all, so the browser — not the server — refuses the response. The anti-DNS-rebinding control is the next row, and it is on by default.

GHOSTDESK_MCP__ALLOWED_HOSTS

localhost,127.0.0.1,::1

Comma-separated Host header allow-list for the MCP endpoint. A request whose Host is not listed gets HTTP 403 — this is what stops a page on an attacker's origin from pointing its own hostname at a locally-running GhostDesk and calling your tools. A deployment reached under a real hostname must name itself here. Do not empty the list.

GHOSTDESK_TLS_CERT

/etc/ghostdesk/tls/server.crt

Path to the TLS certificate. When the file exists, websockify and the MCP server auto-switch to wss:// / https://. See Security.

GHOSTDESK_TLS_KEY

/etc/ghostdesk/tls/server.key

Path to the TLS private key (matching GHOSTDESK_TLS_CERT).

GHOSTDESK_SCREEN__WIDTH

1280

Virtual screen width in pixels. The fallback for a virtual screen with no display to ask — ignored on macOS and Windows, where the display reports its own pixel size.

GHOSTDESK_SCREEN__HEIGHT

1024

Virtual screen height in pixels. Same fallback rule as the row above.

GHOSTDESK_IDLE__TIMEOUT_SECS

1800

Seconds of MCP silence before all open client windows (Firefox, foot, mousepad…) are closed to free memory. Sway, mako, wayvnc and the MCP server itself are spared. Set to 0 to disable — which you want on any native run, macOS or Windows, where the windows it would close are your own.

TZ

America/New_York

IANA timezone (POSIX standard, e.g. Europe/Paris)

LANG

en_US.UTF-8

POSIX locale (e.g. fr_FR.UTF-8)

GHOSTDESK_TLS_CERT / _KEY are the exception that proves the rule: the entrypoint probes those paths to decide the posture, then hands the one it found to the server as GHOSTDESK_HTTP__TLS_CERT_FILE. Everything else you set reaches the server verbatim. A malformed value fails the boot naming the variable rather than silently falling back to a default.

Pinned values (not configurable)

Variable

Value

Rationale

GHOSTDESK_VNC_ADDRESS

127.0.0.1

wayvnc is locked to loopback inside the container's netns; the VNC port is only reachable via the noVNC bridge on 6080. Override attempts are logged and ignored — see SECURITY.md.

Security

GhostDesk owns two things: transport encryption and authentication. Everything else (rate limiting, SSO, WAF, session recording, brute-force protection, per-user identity on noVNC) is a reverse-proxy concern — the container is designed to run behind one, not directly on the internet.

That posture, and the threat model behind it, assume the container. A binary run directly on macOS or Windows has no container boundary to lean on — see either section for what that costs you.

The full threat model, the Auth ≡ TLS posture switch, the wayvnc RFB-type-2-inside-wss:// rationale, the secrets handling contract, and the exhaustive in-scope / out-of-scope table all live in SECURITY.md — single source of truth. Start there before deploying to anything you don't fully trust.

Reporting a vulnerability? Use GitHub's private security advisory — see SECURITY.md § Reporting.

Troubleshooting

My agent's clicks land off-target by a huge margin

Almost always a coordinate-space mismatch. Frontier models (Claude, GPT-4o, Gemini) need no header (default pass-through); the Qwen vision family needs the client to send GhostDesk-Model-Space: 1000 on every MCP request. Full rationale in Model requirements → Coordinate space.

The container refuses to start with a secrets error

The prod posture (cert mounted) requires both GHOSTDESK_AUTH__TOKEN and GHOSTDESK_VNC_PASSWORD to be set — GhostDesk refuses to boot without them on purpose, to prevent an unauthenticated prod container. Generate them as shown in Secure local run and pass them with -e. The demo posture (no cert) has no such requirement.

noVNC shows a black screen or the desktop renders with graphical glitches

You're probably short on shared memory. Browsers and other GPU-accelerated apps inside the container need a reasonable /dev/shm — --shm-size 2g is the baseline in every example and should not be trimmed. If you already have --shm-size 2g, check the container logs for wayvnc or compositor errors.

On macOS, clicks do nothing or screenshots fail

The two privacy permissions are missing. Grant Accessibility (input and windows) and Screen Recording (capture) in System Settings ▸ Privacy & Security, then restart the server — macOS applies the grant at process start. The failing tool names the setting it needs in its error, so read that rather than guessing which of the two it is. If both were working until you rebuilt: the grant is bound to the binary's signature, and a rebuild revokes it. Full walkthrough: Run on macOS.

On Windows, clicks and keystrokes go nowhere

Three causes, and the server's own error names which one. A UAC prompt or the lock screen is in front — the session belongs to Winlogon and no application can drive it, so wait for it to be dismissed. GhostDesk was started as a Windows service — session 0 has no desktop; start it as the signed-in user instead. The target window is elevated — Task Manager, an installer, anything started with Run as administrator — and an unelevated process cannot reach it; everything else on the desktop still works. Full walkthrough: Run on Windows.

On Windows, an application is missing from app_list

The catalogue is the Start Menu, and an entry only counts when its shortcut resolves to an executable that exists. Microsoft Store applications point at a package identity instead, so they are not listed and cannot be launched — install the desktop build of the application if the agent needs to drive it.

Firefox / Electron apps fail to launch or crash immediately

Electron-based apps (VS Code, Slack, Discord…) need Linux user namespaces for their sandbox. Add --cap-add SYS_ADMIN to your docker run (already present in the Secure local run example). Firefox itself works without it.

Build from source

The workspace builds with a plain cargo build. rust-toolchain.toml pins the channel, so rustup resolves the same compiler everyone else has and there is no version to pick. The one native dependency is the WebP encoder, which is C — already buildable on a Linux dev box and with Xcode's command-line tools; on Windows it is the Visual Studio Build Tools with the Desktop development with C++ workload.

git clone https://github.com/YV17labs/GhostDesk.git
cd GhostDesk

cargo build --release        # -> target/release/ghostdesk
cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check

The Justfile wraps those and a few more — just --list prints the set: just check, just lint, just test unit, and just stack, which brings the desktop up (Sway, mako, wayvnc, websockify, the MCP server) under one supervisord. cargo install --path apps/ghostdesk --locked is the same build installed into ~/.cargo/bin, and it is the command the macOS and Windows sections call.

Running what you built, on a Linux host

The container ships a desktop; a Linux binary expects to find one. Give it a Sway session on the Wayland socket for the window seam, grim for capture and wl-clipboard for the clipboard, then:

NESTRS_ENV_PREFIX=GHOSTDESK GHOSTDESK_IDLE__TIMEOUT_SECS=0 target/release/ghostdesk

Both variables are load-bearing for the same two reasons they are on macOS: the prefix is what makes every GHOSTDESK_* name reach the server, and the idle sweep would otherwise close your own windows after thirty minutes of MCP silence. The container exists so that you do not have to assemble that stack — reach for it unless you are working on the Linux backend itself.

The repository

apps/ghostdesk/     the binary: the composition root, and the endpoint's app-local half
crates/features/    one folder per domain — auth, clipboard, host, idle, input,
                    programs, screen — each with its own mcp/ adapter, the only
                    place that knows about the wire
crates/platform/    the OS substrate, and no framework types: input.rs, screen.rs,
                    window.rs, clipboard.rs and desktop.rs are the five contracts,
                    host.rs picks the backend for the compile target, and linux/,
                    macos/ and windows/ are the three that answer them
docker/             base image, services, entrypoint

A fourth desktop is a fourth directory under crates/platform/src/ and no change above it — that is the boundary Built in Rust describes, read from the filesystem.

CONTRIBUTING.md carries the devcontainer setup, the test layout and the PR process; AGENTS.md carries the naming rules the workspace is checked against; CHANGELOG.md records what changed per release.

License

Functional Source License, Version 1.1, ALv2 Future License (FSL-1.1-ALv2) — see LICENSE for the authoritative terms.

What this means in practice (informal summary — the LICENSE file governs; this is not legal advice):

  • Permitted purposes cover the use cases that matter for the vast majority of users: internal use and access inside your company, non-commercial education and research, and professional services you provide to a licensee who is using GhostDesk in accordance with the license. Self-hosting GhostDesk to run your own agents — even commercial, revenue-generating workflows that power your product — is a permitted internal use.

  • Competing Use is prohibited. You may not make GhostDesk available to others in a commercial product or service that substitutes for GhostDesk, substitutes for any product or service the project offers using GhostDesk, or provides the same or substantially similar functionality. In short: you cannot take GhostDesk and rebrand it, host it as a paid service, or build a competing desktop-automation-for-agents product from it.

  • Apache 2.0 in two years. Each released version of GhostDesk becomes available under the Apache License 2.0 on the second anniversary of its release, automatically and irrevocably. The Competing Use restriction only applies for those first two years.

Commercial licensing. If your intended use falls under Competing Use — you want to resell GhostDesk, offer it as a managed service, or build a competing product — contact the maintainers to discuss a commercial license before deploying. Open a GitHub issue or reach out directly; we are happy to talk.

Available Tools

12 tools
app_launchA

Launch a desktop GUI application and return its PID and log file path.

Only applications listed by app_list() are accepted. The process runs in the background; its stdout and stderr are captured in a log file under /tmp/ghostdesk/proc-<pid>.log. Use app_status(pid) to check whether it is still running and to read its output.

Returns a dict with:

  • pid: the process ID of the launched application.

  • log_file: path to the file capturing stdout and stderr.

  • action: description of what was launched.

On failure, returns a dict with a single error key describing what went wrong (not a GUI app, invalid syntax, command not found).

ParametersJSON Schema
NameRequiredDescriptionDefault
commandYes

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: the process runs in the background, stdout/stderr are captured in a specific log file path, and failure returns an error dict. However, it doesn't mention potential side effects like resource consumption or system impact, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and front-loaded, starting with the core purpose. Every sentence adds value: launching details, constraints, behavioral context, return values, and error handling. There is no redundant information, making it efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (launching GUI apps with background processes) and no annotations or output schema, the description is largely complete. It covers purpose, usage, behavior, parameters, and returns. However, it lacks details on permissions, rate limits, or system requirements, which could be relevant for a launch tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning by specifying that the 'command' parameter must be an application listed by app_list(), clarifying its semantics beyond the schema's generic string type. This provides crucial context, though it doesn't detail command format or examples.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Launch a desktop GUI application') and resource ('application'), distinguishing it from siblings like app_list (which lists applications) and app_status (which checks status). It precisely defines what the tool does beyond just the name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidelines: 'Only applications listed by app_list() are accepted' specifies when to use (with listed apps) and implies when not to use (with unlisted apps). It also references alternatives like app_status for checking status, clearly differentiating from sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

app_listA

Return installed GUI apps.

Scans .desktop entries in /usr/share/applications/. Call this before choosing which app to use for a task, or after installing new software during the session.

Returns a list of dicts, each with:

  • name: human-readable application name.

  • exec: the executable to pass to app_launch().

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it scans .desktop entries in /usr/share/applications/, returns a list of dicts with name and exec fields, and clarifies that exec is used with app_launch(). This covers the operational scope and output format well for a read-only tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured with three paragraphs: purpose, usage guidelines, and return format. Each sentence adds value without redundancy. It's front-loaded with the core function and remains appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (0 parameters, no annotations, but has output schema), the description is complete. It explains what the tool does, when to use it, and details the return structure. Since an output schema exists, the description doesn't need to fully document return values, but it still provides helpful semantics for the fields.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters, and schema description coverage is 100% (though empty). The description doesn't need to compensate for any parameter gaps. A baseline of 4 is appropriate since no parameters exist, and the description focuses on the tool's function and output instead.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Return installed GUI apps') and resource (GUI apps from .desktop entries). It distinguishes itself from siblings like app_launch (which launches apps) and app_status (which checks status), establishing a unique purpose for listing applications.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit guidance on when to use this tool: 'before choosing which app to use for a task, or after installing new software during the session.' This gives clear context for usage, though it doesn't explicitly state when NOT to use it or mention alternatives like app_status for checking if an app is already running.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

app_statusA

Check whether a launched app is still running and read its logs.

Only PIDs returned by app_launch() in this session are accepted.

Args: pid: Process ID returned by app_launch(). lines: Number of trailing log lines to return (default 50).

Returns a dict with:

  • pid: the process ID.

  • running: whether the process is still alive.

  • log_file: path to the log file.

  • tail: the last lines lines of stdout/stderr output.

On failure, returns a dict with a single error key.

ParametersJSON Schema
NameRequiredDescriptionDefault
pidYes
linesNo

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it checks running status, reads logs, returns specific data (pid, running, log_file, tail), and handles failures with an error dict. However, it doesn't mention potential side effects, rate limits, or authentication needs, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured and appropriately sized. It starts with the core purpose, provides usage constraints, details parameters with examples, and explains return values. Every sentence adds value without redundancy, and it's front-loaded with essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (monitoring app status), no annotations, no output schema, and 0% schema coverage, the description is largely complete. It covers purpose, usage, parameters, and return values. However, it lacks details on error conditions beyond 'On failure,' and doesn't specify log file formats or access permissions, leaving minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains that 'pid' is a 'Process ID returned by ``app_launch()``' and 'lines' is the 'Number of trailing log lines to return (default 50).' This fully compensates for the schema's lack of descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Check whether a launched app is still running and read its logs.' It specifies the verb ('check' and 'read'), the resource ('launched app'), and distinguishes it from siblings like app_launch (which launches apps) and app_list (which lists apps).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidelines: 'Only PIDs returned by ``app_launch()`` in this session are accepted.' This clearly states when to use (with PIDs from app_launch) and implies when not to use (with other PIDs or outside the session), differentiating it from alternatives like app_list.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

clipboard_getA

Read the current clipboard text.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden. It discloses the read-only behavior ('Read') but doesn't mention potential limitations like platform-specific clipboard access, permissions required, or data format returned. It provides basic behavioral context but lacks depth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it immediately understandable without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (0 parameters, no annotations, but has output schema), the description is mostly complete. It states the purpose clearly, but with no annotations and an output schema present, it could benefit from mentioning what the output contains (e.g., text format) to fully compensate for the lack of behavioral details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters with 100% schema description coverage, so the schema already fully documents the absence of inputs. The description doesn't need to add parameter information, and it correctly implies no parameters are required, earning a baseline score for this scenario.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Read') and resource ('current clipboard text'), distinguishing it from sibling tools like clipboard_set. It precisely defines what the tool does without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context (when clipboard content is needed) but doesn't explicitly state when to use it versus alternatives like screen_shot or other clipboard-related operations. It provides clear intent but lacks explicit comparison guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

clipboard_setA

Write text to the clipboard. Use with key_press("ctrl+v") to paste.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It indicates this is a write operation ('Write text to the clipboard'), which implies mutation, but doesn't specify permissions needed, side effects, or error conditions. It adds some context about integration with key_press, but lacks details on clipboard overwriting or system dependencies.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with two sentences that are front-loaded and waste-free. The first sentence states the core purpose, and the second adds practical usage context, making every word earn its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (1 parameter) and the presence of an output schema (which handles return values), the description is reasonably complete. It covers the action and basic usage, though it could benefit from more behavioral details like error handling or system-specific notes, preventing a perfect score.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It mentions 'text' implicitly but doesn't explain the parameter's purpose, constraints, or format beyond what the schema's title ('Text') provides. The description adds minimal value over the schema, meeting the baseline for low coverage without fully compensating.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Write text to the clipboard') and identifies the resource ('clipboard'), distinguishing it from sibling tools like clipboard_get (which reads) and other UI automation tools. It provides a complete verb+resource statement with no ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description includes explicit guidance on when to use this tool by mentioning 'Use with key_press("ctrl+v") to paste,' which implies it's part of a workflow for pasting operations. However, it doesn't explicitly state when NOT to use it or name alternatives, keeping it from a perfect score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

key_pressA

Press a key or key combination.

Friendly names accepted: Tab, Return, Escape, BackSpace, Left, Page_Up, F4, Ctrl, Alt, Shift, Super. Single printable characters stay as-is (a, c, 5).

Examples: Tab, Ctrl+c, Alt+F4, Ctrl+Shift+Tab.

Returns the standard {action, screen_changed, reaction_time_ms} feedback.

ParametersJSON Schema
NameRequiredDescriptionDefault
keysYes

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses behavioral traits by specifying accepted key names, examples, and return format, but lacks details on permissions, side effects, or error handling. It adds useful context but is not comprehensive for a tool that interacts with system input.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded, starting with the core purpose, followed by details on accepted inputs and examples, and ending with return information. Every sentence adds value without redundancy, making it efficient and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (interacting with system keys) and lack of annotations or output schema, the description is mostly complete: it covers purpose, parameter semantics, and return format. However, it could improve by mentioning potential side effects or error cases, but it's sufficient for basic usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0% description coverage with one parameter 'keys' of type string. The description compensates fully by explaining the semantics: it defines what 'keys' means (key or combination), lists accepted friendly names, and provides examples. This adds significant value beyond the bare schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Press') and resource ('a key or key combination'), and it distinguishes from siblings like 'key_type' (which likely types text) and mouse-related tools by focusing on key presses. The description is precise about what the tool does.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context by listing accepted friendly names and examples, which implicitly guides usage for key presses. However, it does not explicitly state when to use this tool versus alternatives like 'key_type' or other input tools, missing explicit exclusions or comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

key_typeA

Type text. Handles Unicode, newlines, and tabs.

Returns the standard {action, screen_changed, reaction_time_ms} feedback. If screen_changed is false, the text field probably didn't have focus — click on it first and retry.

ParametersJSON Schema
NameRequiredDescriptionDefault
textYes

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden and adds valuable behavioral context: it returns standard feedback (action, screen_changed, reaction_time_ms), explains what screen_changed=false means (text field not focused), and advises retry strategy. It doesn't cover rate limits or error handling, but provides clear operational insight.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with core purpose, followed by return details and troubleshooting advice. Every sentence earns its place: no fluff, efficient structure, and appropriately sized for the tool's complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 1 parameter, no annotations, and no output schema, the description provides good context: purpose, return format, and usage advice. It could mention error cases or limitations, but covers key aspects for a text-input tool. Slightly incomplete but mostly sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It doesn't explicitly mention the 'text' parameter, but implies it through 'Type text' and handling details (Unicode, newlines, tabs). This adds meaning beyond the bare schema, though not fully explicit. For 1 parameter, this is adequate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Type text' with specific capabilities (handles Unicode, newlines, tabs). It distinguishes from siblings like key_press (single key) and clipboard_set (copy-paste), though not explicitly named. The purpose is specific but could be more explicit about sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context through the screen_changed feedback advice: use when a text field has focus, and click first if not. It doesn't explicitly state when to use vs. alternatives like key_press or clipboard_set, nor provide exclusions. Guidance is practical but not comprehensive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_clickA

Click at screen coordinates. Use coordinates from screen_shot() or inspect().

Returns a dict with:

  • action: description of what was performed.

  • screen_changed: whether the 200x200 px zone around the click visibly changed within 2 s. If false the click likely missed its target — retry with adjusted coordinates or take a new screen_shot().

  • reaction_time_ms: how quickly the change was detected (ms).

ParametersJSON Schema
NameRequiredDescriptionDefault
xYes
yYes
buttonNoleft

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it performs a click action, monitors a 200x200 pixel zone for changes within 2 seconds, and provides feedback about success/failure through the screen_changed flag. It also explains what to do when the click misses (retry with adjusted coordinates or take new screenshot).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly structured and concise. It starts with the core action, provides usage guidance, then details the return structure with clear bullet points. Every sentence earns its place, and the information is front-loaded with the most important details first.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (interactive UI operation with feedback), no annotations, and no output schema, the description provides excellent completeness. It explains what the tool does, how to use it, what it returns, and how to interpret the results. The return value documentation compensates for the lack of output schema by clearly describing the response structure and semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description must compensate for the lack of parameter documentation. While it doesn't explicitly document individual parameters, it provides crucial semantic context about parameter usage ('Use coordinates from screen_shot() or inspect()') and the relationship between parameters and tool behavior. The description explains how the x,y parameters relate to screen coordinates and what happens when they're incorrect.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Click at screen coordinates') and distinguishes it from siblings by mentioning coordinate sources from screen_shot() or inspect(). It provides a precise verb+resource combination that differentiates it from other mouse-related tools like mouse_double_click or mouse_drag.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context about when to use this tool ('Use coordinates from screen_shot() or inspect()') and implies when not to use it (when you don't have coordinates). However, it doesn't explicitly contrast with alternatives like mouse_double_click or provide specific exclusion criteria beyond the coordinate requirement.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_double_clickA

Double-click at screen coordinates. Use for opening files or selecting words.

Returns a dict with:

  • action: description of what was performed.

  • screen_changed: whether the 200x200 px zone around the click visibly changed within 2 s. If false the click likely missed its target.

  • reaction_time_ms: how quickly the change was detected (ms).

ParametersJSON Schema
NameRequiredDescriptionDefault
xYes
yYes
buttonNoleft

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it describes the return structure with three specific fields (action, screen_changed, reaction_time_ms), explains what screen_changed=false means ('click likely missed its target'), and specifies the detection zone (200x200 px) and timeframe (2 s). This goes beyond basic parameter documentation to reveal how the tool behaves and interprets results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is perfectly structured and concise: first sentence states purpose with examples, followed by a clear bulleted list of return values with helpful explanations. Every sentence earns its place, with no wasted words or redundant information. It's front-loaded with the core functionality.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (GUI interaction with feedback), no annotations, no output schema, and 0% schema description coverage, the description does well by thoroughly documenting the return behavior and success detection logic. However, it misses some context like coordinate system explanation, error conditions, or performance characteristics. For a mouse interaction tool with rich feedback, it's mostly complete but has minor gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate but provides no parameter-specific information. It doesn't explain what x/y coordinates represent (e.g., screen pixels, relative positioning), the meaning of button options, or default behaviors. The description focuses on output semantics instead, leaving parameters undocumented beyond the schema's basic structure.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Double-click at screen coordinates' with specific examples of use cases ('opening files or selecting words'). It distinguishes from sibling tools like mouse_click (single click) and mouse_drag, but doesn't explicitly contrast with all siblings. The verb+resource combination is specific and actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides implied usage guidance through examples ('Use for opening files or selecting words'), but lacks explicit when-to-use vs. when-not-to-use instructions or named alternatives. It doesn't mention when to choose this over mouse_click or other input tools, leaving some ambiguity about optimal selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_dragA

Drag from one position to another. Use for selecting text, moving items, or resizing.

Returns a dict with:

  • action: description of what was performed.

  • screen_changed: whether the 200x200 px zone around the drop point visibly changed within 2 s.

  • reaction_time_ms: how quickly the change was detected (ms).

ParametersJSON Schema
NameRequiredDescriptionDefault
from_xYes
from_yYes
to_xYes
to_yYes
buttonNoleft

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the action and return values (action description, screen change detection, reaction time), which helps the agent understand the tool's behavior beyond basic parameters. However, it doesn't mention potential side effects like UI changes or system interactions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficiently structured with a clear purpose statement followed by detailed return value documentation. Every sentence adds value: the first explains the action and use cases, the subsequent lines describe the output format. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with no annotations and no output schema, the description provides good behavioral context through return value documentation but completely neglects parameter explanations. The output details help compensate for missing schema, but parameter gaps remain significant.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It doesn't explain any of the 5 parameters (from_x, from_y, to_x, to_y, button) beyond what the schema provides (titles and types). The description focuses on output semantics instead, leaving parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Drag from one position to another') and provides specific use cases ('selecting text, moving items, or resizing'), which distinguishes it from sibling tools like mouse_click or mouse_scroll. However, it doesn't explicitly differentiate from mouse_double_click in terms of dragging vs. double-clicking behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for specific actions (selecting text, moving items, resizing) but doesn't provide explicit guidance on when to use this tool versus alternatives like mouse_click for single clicks or key_press for keyboard interactions. No when-not-to-use scenarios or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

mouse_scrollA

Scroll at a position. direction: up/down/left/right. amount: number of scroll steps (max 5).

Returns a dict with:

  • action: description of what was performed.

  • screen_changed: whether the 200x200 px zone around the scroll point visibly changed within 2 s. If false the page may already be at the scroll boundary.

  • reaction_time_ms: how quickly the change was detected (ms).

ParametersJSON Schema
NameRequiredDescriptionDefault
xYes
yYes
directionNodown
amountNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the action (scroll at a position), constraints (max 5 steps), and return values including screen change detection and reaction time, which adds valuable context beyond basic parameters. However, it doesn't cover potential errors or side effects like out-of-bounds scrolling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is appropriately sized and front-loaded, starting with the core action and key parameters. Every sentence adds value: the first defines the tool, the second explains parameters, and the third details return values, with no wasted words or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (interactive UI tool), no annotations, and no output schema, the description is fairly complete. It covers the action, parameters, constraints, and return structure, but could improve by addressing error cases or integration with sibling tools like screen_shot for verification.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining 'direction' as up/down/left/right and 'amount' as number of scroll steps with a max of 5, which clarifies beyond the schema's enum and integer types. However, it doesn't detail 'x' and 'y' parameters (e.g., coordinate system or units).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose with a specific verb ('Scroll') and resource ('at a position'), and distinguishes it from siblings like mouse_click or mouse_drag by focusing on scrolling behavior. It explicitly mentions the direction and amount parameters, making the action distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like mouse_drag or key_press for navigation, nor does it mention prerequisites such as needing a visible screen or active application. It lacks explicit when/when-not instructions or sibling comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

screen_shotA

Capture the screen, optionally cropped to a region.

Args: region: Area to capture (full screen if omitted). format: "webp" (default, smaller payload) or "png" (lossless). stabilize: Wait for the page to stop moving before capturing (max 2.5 s). Useful right after navigation.

ParametersJSON Schema
NameRequiredDescriptionDefault
regionNo
formatNowebp
stabilizeNo

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by explaining the stabilization behavior ('Wait for the page to stop moving before capturing'), timeout constraint ('max 2.5 s'), and default behavior ('full screen if omitted'). However, it doesn't mention what happens on failure or the output format beyond format options.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Perfectly structured with a clear opening statement followed by organized parameter explanations. Every sentence adds value: the first establishes purpose, and each parameter description provides essential context without redundancy. The formatting with bullet-like indentation enhances readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 3-parameter tool with no annotations and no output schema, the description does an excellent job covering parameter semantics and basic behavior. The main gap is lack of information about what the tool returns (image data format, error conditions), which would be important given the absence of output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description fully compensates by explaining all three parameters: 'region' (area to capture, full screen default), 'format' (webp vs png with rationale), and 'stabilize' (behavior and use case). Each parameter gets meaningful context beyond what the bare schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the specific action ('Capture the screen') and distinguishes it from all sibling tools (which are about app control, clipboard, and mouse/keyboard interactions). It provides a clear verb+resource combination that is unambiguous in this context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use certain features ('useful right after navigation' for stabilize parameter), but doesn't explicitly state when to use this tool versus alternatives. Since sibling tools are all different interaction types (not alternative screenshot methods), this is reasonable, but no explicit comparison is made.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 12 tool updatesv7.0.1
    • First observedapp_launch
    • First observedapp_list
    • First observedapp_status
    • First observedclipboard_get
    • First observedclipboard_set
    • First observedkey_press
    • First observedkey_type
    • First observedmouse_click
    • First observedmouse_double_click
    • First observedmouse_drag
    • First observedmouse_scroll
    • First observedscreen_shot
Install Server
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

TDQS

A4.2/5.0
Disambiguation5/5

Each tool has a clearly distinct purpose with no overlap: app_launch, app_list, and app_status form a coherent app management group; clipboard_get/set handle clipboard operations; key_press and key_type cover keyboard input; mouse_click, mouse_double_click, mouse_drag, and mouse_scroll provide distinct mouse actions; and screen_shot handles screen capture. The descriptions clearly differentiate their functions, preventing misselection.

Naming Consistency4/5

The naming is mostly consistent with a verb_noun pattern (e.g., app_launch, clipboard_get, mouse_click), but there are minor deviations: key_press and key_type use 'key' instead of 'keyboard', and screen_shot uses 'shot' instead of 'capture'. These deviations are minor and do not significantly hinder readability or predictability.

Tool Count5/5

With 12 tools, the count is well-scoped for a desktop automation server covering app management, clipboard, keyboard, mouse, and screen operations. Each tool earns its place, providing a comprehensive yet manageable set for the domain without being overly sparse or bloated.

Completeness5/5

The tool set offers complete coverage for desktop automation: app management (launch, list, status), clipboard operations (get/set), keyboard input (press/type), mouse actions (click, double-click, drag, scroll), and screen capture. There are no obvious gaps; agents can perform full workflows from launching apps to interacting with them via input and monitoring via screenshots.

Maintenance

ActivityMaintained
ResponsivenessUnresponsive

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • taw-computer

    tawgroup
    A
    license
    B
    quality
    D
    maintenance
    An MCP server that provides AI agents with a full Ubuntu desktop environment inside Docker, enabling them to perform complex computer tasks like browsing, coding, testing, and GUI automation.
    36
    8
    MIT

  • umbriel

    ObscuritySRL
    A
    license
    Not graded
    quality
    B
    maintenance
    MCP server for driving any Windows app through five layers including OCR, UI Automation, and direct OS operations. Enables AI agents to control Windows desktop and OS cursor-free, even on background/locked windows.
    164
    2
    MIT

  • da-mcp

    cioinside
    F
    license
    A
    quality
    A
    maintenance
    Cross-platform desktop automation MCP server that lets AI agents capture screenshots, run OCR with UI-element classification, control mouse/keyboard, and launch programs on Linux, macOS, and Windows.
    20
    -

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/YV17labs/GhostDesk'

If you have feedback or need assistance with the MCP directory API, please join our Discord server