Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type @ followed by the MCP server name and your instructions, e.g., "@x11-mcp open Firefox on the desktop and screenshot the homepage"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
An MCP server that gives an agent its own computer to use: a private, headless X11 desktop (Xvfb + Openbox) running next to yours. The agent launches apps, takes screenshots, and drives the mouse and keyboard there. None of it reaches your session: your cursor stays put, your focus is never taken, nothing appears on your screen, and you don't need to be on any particular workspace.
The agent's desktop is a separate X server with no physical input, so this works from any desktop (GNOME, KDE, Sway, Hyprland, an X11 session, ...) or none at all, e.g. over ssh or in CI.
Apps run on their X11 backend, so this tests an app's X11 path, not its Wayland one.
The flake bundles everything (Xvfb, Openbox, dbus-daemon, TigerVNC's Xvnc, x11vnc):
claude mcp add --scope user x11-desktop -- nix run github:Anti-2k/x11-mcpOr build a local copy: git clone https://github.com/Anti-2k/x11-mcp && cd x11-mcp && nix build, then point your client at result/bin/x11-mcp.
Install the X server, the window manager and D-Bus. Xvnc and x11vnc
are optional (see Watching the agent).
Distro | Required | Optional |
Arch |
|
|
Debian, Ubuntu |
|
|
Fedora |
|
|
openSUSE |
|
|
Gentoo |
|
|
Void |
|
|
Alpine |
|
|
Guix |
|
|
Package names drift; repology has the current ones.
Install the server (Python 3.10+) with uv or
pipx. Its Python dependencies (mcp, python-xlib, Pillow) come as wheels.
uv tool install git+https://github.com/Anti-2k/x11-mcp
# or: pipx install git+https://github.com/Anti-2k/x11-mcpCheck that everything is found, then register it:
x11-mcp check
claude mcp add --scope user x11-desktop -- x11-mcpAny other MCP client: run x11-mcp as a stdio server. The desktop starts on
the first tool call and is torn down, along with every app the agent started,
when the server exits.
Variable | Default | Meaning |
|
| Desktop name. A second server with the same name gets |
|
| Size of the headless screen. |
|
|
|
|
| Number of workspaces at start (1-32). The agent can change it with |
| unset |
|
| unset | Comma-separated variables to pass through anyway, e.g. |
|
| Use a different Openbox binary. |
Related MCP server: WindowsPC-MCP
Tool | |
| Screen size, workspaces (which is shown, window counts), focused window |
| Whole screen or a region; |
| A region magnified (up to 4x) to read small text |
| Run a shell command on the desktop, optionally on a given workspace; focuses and reports the new window |
| Run a short command in the desktop's environment and return its output (killed after |
| Windows topmost first: id, title, class, pid, workspace, frame and content geometry, focused/maximized/fullscreen/minimized |
| Window management ( |
| Place a window; arrange a workspace as |
| Workspaces, numbered from 1 |
| Pointer, in screen pixels |
| Keyboard: any Unicode text regardless of keymap; combos like |
|
|
Windows float: they keep the size and position they open with until something
moves them, so coordinates from a screenshot stay valid when the agent clicks.
tile_windows gives side-by-side tiling on demand.
A hidden workspace has no pixels (Openbox unmaps its windows), so
screenshot(workspace=N) briefly switches the view there, waits for the apps to
repaint, captures, switches back and restores the focus.
wait_until_idle watches the whole screen with the X DAMAGE extension.
Redrawn rectangles smaller than min_area (default 400 px², about 20x20: blinking
text cursors, small spinners) don't count, so a blinking cursor doesn't keep it
waiting forever.
x11-mcp watch agent # view-only VNC server on localhost; connect any viewer to the PORT it prints
eval "$(x11-mcp env agent)" # point this shell at the agent desktop: exports DISPLAY and XAUTHORITYwatch runs x11vnc -viewonly -localhost -nosel against the agent's display.
With X11_MCP_SERVER=xvnc the X server is the VNC server: watch then prints
its port (5900 + display number). It runs with viewer keyboard, pointer,
resize and clipboard input all disabled. Either way, a viewer can look but
cannot type, click or move the agent's pointer (the live test checks this).
Both listen on 127.0.0.1 without a password, so on a machine shared with other
users, they can watch too while it runs.
The agent gets its own X server rather than a corner of yours. In one X session there is one pointer and one keyboard focus, so any synthetic click or keystroke would have to move your cursor and take your focus.
Way the two sides could interfere | What prevents it |
Your keyboard and mouse reaching the agent | The agent's Xvfb (or Xvnc) is headless with no input devices. Xvnc and |
Agent input reaching your windows | Input goes in through the XTEST extension on the agent's own X server. Nothing uses |
Agent apps opening on your screen | Every process gets a freshly built environment whose only display is the agent's ( |
Other programs connecting to the agent's display | The X server runs with |
The shared | Both X servers have their sockets there (plus abstract ones, |
Single-instance apps handing off to your copy | A private D-Bus session bus, plus by default a separate |
Notifications, portals and other D-Bus services showing up on your desktop | The private bus. Services activated on it inherit the agent's display. |
Your session's environment being rewritten | Openbox is started directly (not |
Window manager bindings | The agent's Openbox config has no key bindings, no root-window or desktop mouse bindings and no autostart. The only mouse bindings are the ones a window frame needs (click to focus, drag the title bar or edges, the title bar buttons), none with a modifier, so no app shortcut is intercepted. VT-switch, server-kill and grab-breaking keysyms are refused. |
Two agents fighting over one desktop | Each server takes an exclusive lock on its desktop name. |
Leftover processes | Every |
tests/test_live.py checks this end to end:
It starts a desktop and drives terminals on two workspaces.
While they run, it asserts (from ss -x socket peers) that no agent process holds a connection to your X server or Wayland compositor.
Afterwards, it asserts that your systemctl --user show-environment and the contents of your runtime directory are unchanged.
This keeps the agent and you from getting in each other's way. It is not a security sandbox. Agent processes run as your user, with your files and network.
Your display: a process that deliberately sets DISPLAY=:0, or
WAYLAND_DISPLAY=wayland-0 XDG_RUNTIME_DIR=/run/user/$UID, can still reach
your session. The abstract X socket can't even be hidden with a mount namespace;
that takes a network namespace too.
gpg: gpg finds its agent at a fixed path, so signing inside the agent
desktop can pop a pinentry dialog on your screen.
Sound: agent apps have none. The private runtime directory has no PipeWire or PulseAudio socket.
Graphics: Xvfb renders OpenGL in software (Mesa llvmpipe). GPU-heavy apps work, but slowly.
Covering the first two needs namespaces (e.g. bubblewrap). That is not done here.
With X11_MCP_SHARE_HOME=1, an app that is already running on your desktop
(Chromium, VS Code, Discord, ...) may receive the agent's launch and open the
window on your screen. To use those apps with a shared home, give them a
separate profile (--user-data-dir, firefox --profile ... --new-instance).
MIT, see LICENSE.
nix develop
python -m pytest # unit tests
X11_MCP_LIVE=1 python -m pytest # plus the live end-to-end test (needs xterm)
X11_MCP_LIVE=1 X11_MCP_SERVER=xvnc python -m pytest # the same against XvncLogs: ~/.local/state/x11-mcp/server.log; the per-desktop xserver.log,
openbox.log, dbus.log and per-app logs live in the desktop's runtime
directory ($XDG_RUNTIME_DIR/x11-mcp/<name>) while it runs.
Run a shell command inside the agent desktop's environment and return its output.
For short commands: xclip/xdotool, reading app logs, files under the agent's $HOME.
It sees the agent desktop's display and D-Bus, not the user's. Use launch for
GUI apps and anything long-running: this waits for the command, and kills it and
everything it started after timeout seconds (max 300). stdin is empty.
Inputs: command (required), cwd, timeout. Returns: result
Click at (x, y), or where the pointer is. count=2 double-clicks, 3 triple-clicks.
Inputs: x, y, count, button. Returns: result
Ask a window to close, as its close button would. The app may ask to confirm.
Inputs: window_id (required). Returns: result
Screen size, workspaces (* = shown, (n) = window count), focused window and how the user can watch.
Inputs: no named parameters. Returns: result
Press at (x1, y1), move to (x2, y2) in steps, release.
Inputs: x1 (required), x2 (required), y1 (required), y2 (required), +1 more. Returns: result
Focus and raise a window (switching to its workspace and unminimizing it if needed).
Inputs: window_id (required). Returns: result
Run a shell command on the agent desktop (e.g. 'xterm', 'firefox --new-instance').
The window opens on workspace (the view switches there first) or the current one.
Waits up to wait_for_window seconds (max 60) for the new window, focuses it and
reports it.
Inputs: command (required), cwd, workspace, wait_for_window. Returns: result
All windows (or those on one workspace), topmost first, with ids for the window tools.
frame = the whole window on screen (title bar and borders included), content = the app's own area; both are (x,y widthxheight) in screen pixels.
Inputs: workspace. Returns: result
Move the agent's pointer to screen pixel (x, y).
Inputs: x (required), y (required). Returns: result
Place a window's frame (title bar and borders included) at (x, y) with this outer size.
Apps with size steps (terminals snap to character cells) or limits may end up a little smaller; the result reports the actual geometry.
Inputs: x (required), y (required), width (required), height (required), +1 more. Returns: result
Move a window (default: the focused one) to another workspace. The view stays where it is.
Inputs: workspace (required), window_id. Returns: result
Press key combos, space-separated: 'ctrl+l', 'Return', 'ctrl+shift+t alt+F4'.
Key names are X keysyms (Return, Tab, Escape, BackSpace, Up, F5, a, 1, ...).
Inputs: keys (required), repeat. Returns: result
Kill the agent desktop and every app on it, then start a fresh one.
Inputs: no named parameters. Returns: result
Capture the agent desktop, or a region of it (x, y, width, height).
Use scale < 1 to save tokens; click coordinates are always full-scale screen pixels.
workspace captures another workspace: the view switches there for the capture and
then back (hidden workspaces have no pixels until shown).
Inputs: x, y, scale, width, +4 more.
Scroll by wheel notches at (x, y) or the pointer. Negative down = up, negative right = left.
Inputs: x, y, down, right. Returns: result
Change how many workspaces there are (1-32). Windows on removed ones move to the last remaining.
Inputs: count (required). Returns: result
Arrange the normal windows of a workspace (default: the shown one) side by side.
columns: next to each other; rows: stacked; grid: rows of up to ceil(sqrt(n)). Windows go in the order they were opened. Dialogs and minimized windows are left alone.
Inputs: layout, workspace. Returns: result
Type text into the focused window. Any Unicode works, independent of keyboard layout.
Newlines press Return and tabs press Tab.
Inputs: text (required), delay_ms. Returns: result
Show a workspace (numbered from 1). Windows on other workspaces are hidden, not closed.
Inputs: workspace (required). Returns: result
Sleep, e.g. to let an app finish loading before a screenshot. Max 30s.
Inputs: seconds (required). Returns: result
Wait until nothing on screen has been redrawn for quiet_ms (or timeout seconds pass, max 60).
Each redrawn rectangle smaller than min_area square pixels (default about 20x20:
blinking text cursors, small spinners) is ignored; min_area=0 counts every pixel.
Inputs: timeout, min_area, quiet_ms. Returns: result
Apply an action to a window (default: the focused one).
kill disconnects the window's app from the display (XKillClient), which ends most apps immediately, and kills its process if that survives; use close_window first.
Inputs: action (required), window_id. Returns: result
Magnify a region (x, y, width, height) of the screen to read small text or icons.
The image is upscaled by factor (default: to about 1568px on the long side, at most 4x).
Image pixel (px, py) is screen pixel (x + px / factor, y + py / factor).
Inputs: x (required), y (required), width (required), height (required), +1 more.
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
Scored across 23 tools
Most tools have clearly distinct resource/action targets: screenshot vs zoom vs desktop_info, and bash vs launch are separated by purpose and description. Minor ambiguity remains around window_action (a generic escape hatch that overlaps close_window's kill behavior) and wait vs wait_until_idle, though descriptions clarify both.
All names use consistent snake_case, and the dominant pattern is verb_noun (focus_window, close_window, move_window_to_workspace, type_text, press_keys). A few outliers like mouse_move (noun_verb) and noun-style names (screenshot, bash, desktop_info) slightly break the pattern but remain readable.
23 tools is on the heavy side, but the domain spans screenshots, input, windows, workspaces, and process control, so most tools earn their place. A few could conceivably be merged (e.g. wait/wait_until_idle), but the set is not bloated relative to scope.
The surface covers the desktop lifecycle well: capture, input, window management, workspace management, process launch, and restart. Gaps include no explicit right/middle-click button parameter, no maximize/minimize window actions, and no dedicated clipboard tool, though bash/xdotool can work around these.
Cloud desktops for AI agents: create a Linux computer, see its screen, click, type and run tasks.
A cloud Linux desktop for your AI: browse, click, type, read the screen and run commands.
Real desktops for AI agents, on machines you choose: screenshot, type, click, hire more agents.
Persistent Linux computers for AI agents: desktop, signed-in browser, vault, apps and shell.