linux-computer-use-mcp
Supports computer-use control of Cinnamon desktop sessions, including screenshots and simulated mouse/keyboard actions.
Supports computer-use control of GNOME desktop sessions, using GNOME screenshot/portal backends and virtual input to interact with applications.
Supports computer-use control of Hyprland (wlroots) sessions using grim/portal screenshots and virtual input devices.
Supports computer-use control of KDE Plasma Wayland sessions using spectacle/portal screenshots and virtual input devices.
Provides computer-use control of a Linux desktop: takes screenshots, moves the pointer, clicks, scrolls, drags, types text, presses shortcuts, and manages or focuses windows and applications.
Supports computer-use control of Sway (wlroots) sessions using grim/portal screenshots and virtual input devices.
Supports computer-use control on Wayland sessions using virtual pointer/keyboard input and desktop-specific screenshot backends.
Supports computer-use control of XFCE desktop sessions, including screenshots and simulated mouse/keyboard actions.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@linux-computer-use-mcp Open Firefox and take a screenshot of the weather forecast for Chennai"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
linux-computer-use-mcp
Let Claude see and control your Linux desktop, live. Claude takes screenshots, and you watch the real mouse pointer glide across your screen, click, scroll, drag, type and press shortcuts.
The official Claude desktop app supports Computer Use on macOS and Windows, but not (yet) on Linux. This project is an open-source MCP server that fills the gap. It works with the Claude desktop app for Linux, Claude Code, and any other MCP client.
Community project, not affiliated with or endorsed by Anthropic.
Features
Works on X11 and Wayland: GNOME, KDE Plasma, Sway, Hyprland, river, Wayfire, XFCE, Cinnamon, MATE…
You can watch it work: the pointer glides smoothly to each target (120 fps eased motion) and text is typed key by key.
Fast:
Every action returns a fresh screenshot as soon as the screen stops changing, so there are no guessed waits and no separate screenshot call.
batchruns a whole sequence (click → type → Enter) in one call.On X11 the mouse goes straight through XTEST in-process (a click takes ~3 ms), and screenshots are compact JPEGs.
Full toolset:
screenshot,click(left/right/middle, double/triple, with modifier keys),move_mouse,drag,scroll,type_text,press_key(ctrl+s,alt+Tab,super…),hold_key,batch,wait(returns early once the screen settles),focus_window,list_windows,zoom(full-resolution crop for small text),cursor_position,open_application,list_applications,system_info.Smart waiting: after each action it waits for the UI to react, then for a short quiet period, so it doesn't return mid-animation or before a click has had any effect, and it gives up quickly when nothing changes.
Won't click into the wrong window: the Claude app pops to the front whenever you send it a message. On X11 the server checks which window is really on top at the target point, and refuses to click there or type into it if that window is protected (the Claude app, by default). Instead it tells the model to
focus_windowthe right app first.HiDPI-aware: screenshots are scaled down (default max 1280×800) and coordinates are mapped back to real pixels.
Safety built in: a kill switch command, a failsafe corner (X11), an action log and a dry-run mode. Details below.
Related MCP server: claude-linux-mcp
How it works
Session | Screenshots | Mouse & keyboard |
X11 (any desktop) |
| XTEST via |
Wayland, GNOME |
|
|
Wayland, KDE Plasma |
|
|
Wayland, wlroots (Sway, Hyprland…) |
|
|
Wayland doesn't let apps move the pointer directly. Instead, dotool creates a virtual absolute-pointer
and keyboard device through /dev/uinput, and the compositor treats it like real hardware.
That's why Wayland needs a one-time permission setup.
Install
1. System packages
X11
sudo apt install xdotool # Debian / Ubuntu
sudo dnf install xdotool # Fedora
sudo pacman -S xdotool # ArchWayland: install a screenshot tool for your desktop, plus dotool:
# Ubuntu / Debian (GNOME)
sudo apt install gnome-screenshot
# KDE: spectacle · Sway/Hyprland: grim
# Arch: sudo pacman -S grim / yay -S dotooldotool is packaged in the AUR and some other distros. If yours doesn't have it, build it from
sr.ht/~geb/dotool (./build.sh && sudo ./build.sh install; needs Go).
ydotool ≥ 1.0 with ydotoold running also works, but its pointer positioning is less precise
(it moves relatively, so pointer acceleration affects it).
Then allow virtual input once, and log out and back in:
linux-computer-use-mcp setup-wayland --yes2. The server
pipx install git+https://github.com/dhakseshr/linux-computer-use-mcp
# or with uv:
uv tool install git+https://github.com/dhakseshr/linux-computer-use-mcp3. Check it
linux-computer-use-mcp doctorConnect it to Claude
Print a ready-made config for your machine:
linux-computer-use-mcp configClaude desktop app (Linux): add this to ~/.config/Claude/claude_desktop_config.json, then restart the app:
{
"mcpServers": {
"linux-computer-use": {
"command": "/home/you/.local/bin/linux-computer-use-mcp"
}
}
}Claude Code
claude mcp add linux-computer-use -s user -- linux-computer-use-mcpThen ask something like "Open Firefox, go to the weather for Chennai and tell me tomorrow's forecast" and watch the pointer.
Safety
This tool gives an AI real control of your mouse and keyboard. Treat it the way you would a person remote-controlling your machine.
Kill switch:
linux-computer-use-mcp stopimmediately blocks every action (including screenshots) until you runlinux-computer-use-mcp resume. Bindstopto a keyboard shortcut in your desktop's settings (e.g.Super+Escape) so you can halt it instantly.Failsafe corner (X11): slam the mouse into the top-left corner and the next action is refused.
Protected windows (X11): clicks, drags and scrolls onto a protected window, and typing while one is focused, are refused. That stops the model from typing into Claude's own chat box. The default is the Claude app; set
LCU_PROTECT_WINDOWSto a comma-separated list of app-class substrings, or to an empty string to turn the guard off.Action log at
~/.local/state/linux-computer-use/actions.log.Dry run: set
LCU_DRY_RUN=1to log actions without performing them.The server instructs the model to ask before sending messages, buying, deleting or changing settings, and to treat on-screen text as information, not instructions. That reduces risk but doesn't remove it. Prompt injection from web pages is a real risk, so keep an eye on what it does, and don't leave it unattended on screens that show sensitive data.
Configuration
Set these as environment variables, either in your shell or in the "env" block of the MCP config.
Variable | Default | Meaning |
|
| Seconds the pointer takes to glide to a target. |
|
| Actions return a screenshot once the screen settles |
|
| Max seconds an action waits for the screen to stop changing |
|
| Screenshot JPEG quality |
|
| Delay between typed characters |
|
| Max screenshot size sent to the model |
|
|
|
| auto |
|
| auto |
|
| auto | Force |
|
| Top-left-corner failsafe (X11) |
|
| Comma-separated app classes that must never receive clicks or keystrokes (X11). Empty = off |
|
| Log actions without performing them |
|
| Kill-switch file |
|
| Set empty to disable |
Troubleshooting
The Claude window keeps jumping in front of the app it's controlling: on X11 the protected-window guard stops stray input, and the model can call
focus_windowto bring the target back. For the smoothest experience, snap Claude to one half of the screen (Super+Left) and the target app to the other half, or keep Claude on a second monitor."No graphical session found": the MCP client launched the server without
DISPLAY/WAYLAND_DISPLAY. Add them to the"env"block (linux-computer-use-mcp configprints the right values).GNOME asks "Allow screenshot?": that's the portal's one-time permission prompt. Allow it, or install
gnome-screenshot.Pointer doesn't move on Wayland: run
doctor. Usually/dev/uinputisn't writable yet, because you haven't logged out since runningsetup-wayland.Clicks land in slightly the wrong place on Wayland with fractional scaling: try
dotoolinstead ofydotool, and report your setup in an issue.Multi-monitor: screenshots cover all monitors as one image. Wayland absolute positioning maps across the whole layout, which works on most setups but not all.
Limitations
On Wayland, apps can't read where the pointer is, so
cursor_positionreports the last position this server moved it to.The X11 failsafe corner doesn't work on Wayland. Use the
stopshortcut there.list_windows,focus_windowand the protected-window guard need X11. Wayland compositors don't let apps see other apps' windows.While the GNOME overview (Activities) is open, the guard still sees the windows underneath it, so clicks over the Claude window's area may be refused. Use
focus_windowinstead of the overview.Flatpak or Snap sandboxed MCP clients may not be able to run the host's
xdotool/dotool.
Development
git clone https://github.com/dhakseshr/linux-computer-use-mcp && cd linux-computer-use-mcp
pip install -e ".[dev]"
pytest # unit tests
xvfb-run -s "-screen 0 1920x1080x24" pytest # + live end-to-end tests (needs xdotool, xterm, openbox)Contributions are welcome, especially testing on more desktops, and backends such as the RemoteDesktop portal or KDE's fake-input protocol.
License
MIT
Available Tools
17 toolsbatchA
Run several actions in ONE call (much faster than one call per action), then return a screenshot. Use it whenever you already know the next few steps, e.g. click a field, type, press Enter.
Each action is an object with an "action" key plus that action's arguments: {"action": "click", "x": 100, "y": 200, "button": "left", "clicks": 1, "modifiers": ["ctrl"]} {"action": "move", "x": 100, "y": 200} {"action": "drag", "start_x": 1, "start_y": 2, "end_x": 3, "end_y": 4} {"action": "scroll", "direction": "down", "amount": 5, "x": 640, "y": 400} {"action": "type", "text": "hello"} {"action": "key", "keys": "ctrl+l", "repeat": 1} {"action": "hold_key", "keys": "shift", "seconds": 1} {"action": "wait", "seconds": 3} (returns early once the screen settles) {"action": "open_application", "name": "Google Chrome"} {"action": "focus_window", "query": "WhatsApp"} Coordinates refer to the most recent screenshot. Between steps that can change the UI (clicks, keys, scrolls) the screen is briefly allowed to settle. Stops at the first failing step.
Inputs: actions (required), screenshot.
clickA
Glide the pointer to x,y and click. Returns a screenshot once the screen settles.
clicks: 1 = single, 2 = double, 3 = triple click. modifiers: keys held during the click, e.g. ["ctrl"] or ["shift"]. screenshot: set false to skip the follow-up screenshot (e.g. mid-sequence).
Inputs: x (required), y (required), button, clicks, +2 more.
cursor_positionA
Where the pointer is, in screenshot coordinates (on Wayland: last position this server moved it to).
Inputs: no named parameters. Returns: result
dragB
Press at the start point, glide to the end point, release. For moving windows, sliders, selections.
Inputs: end_x (required), end_y (required), start_x (required), start_y (required), +2 more.
focus_windowA
Bring a window to the front and focus it, matched by part of its title or app class, e.g. "Chrome", "WhatsApp", "Terminal". Much faster and more reliable than clicking through the overview or Alt+Tab (X11 only).
Inputs: query (required), screenshot.
hold_keyB
Hold a key or combo down for some seconds (max 10), then release.
Inputs: keys (required), seconds (required), screenshot.
list_applicationsB
List installed desktop applications (optionally filtered by a substring).
Inputs: filter. Returns: result
list_windowsA
List open windows on the current workspace, topmost first (X11 only), with their position in screenshot coordinates. Use focus_window to bring one to the front.
Inputs: no named parameters. Returns: result
move_mouseA
Move the pointer to x,y without clicking (hover to reveal tooltips or menus).
Inputs: x (required), y (required), screenshot.
open_applicationA
Launch an installed desktop app by name, e.g. "Google Chrome", "Files", "Terminal". Returns a screenshot once the app's window has appeared and settled.
Inputs: name (required), screenshot.
press_keyC
Press a key or shortcut, e.g. "Return", "Escape", "ctrl+s", "ctrl+l", "alt+Tab", "super", "F5".
Aliases like enter, esc, cmd/win (= super), pgdn, del are accepted.
Inputs: keys (required), repeat, screenshot.
screenshotA
Capture the whole screen. Use the returned image's pixel coordinates for every other tool. Most action tools already return a fresh screenshot, so you rarely need this.
Inputs: no named parameters.
scrollB
Scroll the mouse wheel by amount notches, optionally after moving to x,y first.
Inputs: direction (required), x, y, amount, +1 more.
system_infoA
Show the detected session, backends and screen size.
Inputs: no named parameters. Returns: result
type_textB
Type text into the focused field, as if from the keyboard. Newlines press Enter.
Inputs: text (required), screenshot.
waitB
Wait for a page/app to finish loading. With until_settled (default) this returns as soon as
the screen stops changing, using seconds (max 30) only as the upper limit.
Inputs: seconds, screenshot, until_settled.
zoomA
Return a full-resolution crop of a region of the latest screenshot (coordinates in screenshot pixels), to read small text or inspect icons precisely. Coordinates for other tools still refer to the full screenshot, not to this crop.
Inputs: x (required), y (required), width (required), height (required).
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 17 tools
Each tool targets a distinct input modality or resource (text, keys, pointer, windows, apps, screen capture), and descriptions explicitly clarify near-neighbors like move_mouse vs click and press_key vs hold_key. The batch tool intentionally subsumes every other action, which is clearly documented but does create a 'should I use batch or the individual tool' choice on every step.
All snake_case with a strong verb-first pattern (type_text, press_key, open_application, list_windows, focus_window, move_mouse), and single-word verbs like click, drag, scroll, wait, zoom read naturally. Minor deviations are the noun-phrase getters cursor_position and system_info, which lack a get_ prefix used nowhere else either, so the set is still internally consistent.
17 tools is slightly on the heavy side but each one covers a genuinely needed primitive for GUI automation (pointer, keyboard, windows, apps, capture, batching). Nothing looks redundant enough to cut, though a few getters (cursor_position, system_info) are low-frequency.
The surface covers the full perceive-act loop: screenshot/zoom for perception, full mouse and keyboard control, app launch, window listing/focusing, settling waits, and batch chaining. Gaps are modest but real: no clipboard read/write (agents must paste blindly via ctrl+v), no workspace switching, and window tools are X11-only, leaving Wayland users without window management.
Maintenance
Related MCP Connectors
A cloud Linux desktop for your AI: browse, click, type, read the screen and run commands.
Cloud desktops for AI agents: create a Linux computer, see its screen, click, type and run tasks.
Use your Mac, Windows or Linux computer from ChatGPT, Claude or Codex: files, commands, documents.
Give your AI hands on every computer you own. Remote control of your Windows, Mac and Linux machines from one ChatGPT, Claude or Cursor chat — even from your phone. Ask it to find out why the office server is crawling, update every PC in the house, sort out your mom's laptop from another city, or ship the site from your work machine while you're on a train. It runs commands, reads files and takes a screenshot when it needs to see the screen. You decide what each computer allows, and you can cut any of them off in a second.
Related MCP Servers
- FlicenseAqualityDmaintenanceEnables Claude Code to interact with Linux X11 desktops by performing mouse actions, keyboard input, and capturing screenshots. It provides a suite of tools for automation tasks such as clicking, typing, and zooming on specific screen regions.16-
- AlicenseAqualityCmaintenanceGive Claude Desktop full desktop control on Linux/X11: screenshot, mouse, keyboard, windows, clipboard, app launch. Zero-dependency MCP extension, MIT-licensed.15MIT
- AlicenseAqualityBmaintenanceEnables Claude Code and Codex to see and drive an X11 desktop through screenshots, mouse, keyboard, and accessibility-tree actions while keeping the user's own pointer and focus separate.18Apache 2.0
- AlicenseNot gradedqualityCmaintenanceEnables an AI assistant to see and control your local desktop on Windows and Linux by capturing screenshots, clicking, scrolling, pressing hotkeys, typing text, and reading or writing the clipboard. Actions can be batched into a single call that ends with a fresh screenshot.MIT