Linux Desktop Control

Let Codex see and control apps on your Linux desktop through a local MCP server. It can click, type, scroll, drag, and paste text, with Stop controls in a tray menu or a fallback panel so you can take back control.

Beta 0.1.0b3: tested on Ubuntu GNOME, X11, and Python 3.12. Wayland is not supported. Other setups and MCP clients are unverified. This is an independent project, available under the MIT license.

Get started

You need an unlocked Ubuntu GNOME desktop running X11. Check your session:

echo "$XDG_SESSION_TYPE"

It should print x11. XWayland inside a Wayland session is not enough.

On Ubuntu 24.04 with Python 3.12, install the prerequisites and register with Codex:

sudo apt-get update
sudo apt-get install python3-venv python3-gi gir1.2-gtk-3.0 libglib2.0-bin

git clone https://github.com/KingAkeem/linux-desktop-control.git
cd linux-desktop-control
./install --register

The installer downloads pinned dependencies into .venv. Registration adds an MCP server, a skill link, and a marked section in AGENTS.md under ~/.codex (or CODEX_HOME if set). Review those changes, then reload MCP servers or restart Codex. Keep the checkout in place while registered.

Related MCP server: mcp-computer-use

Try it

Open an empty text editor and ask Codex:

Use Linux Desktop Control to select the empty editor, type Hello Linux, capture the result, and stop.

Check the screenshot to confirm the result. This uses your current desktop and shares your pointer and keyboard. Stay present while it runs.

To stop: move your mouse, press a key, click Stop, or use Ctrl+Alt+Escape. Detection is reactive, so your first input may still reach the app. A latched Stop needs a local Resume permission click before a new task can start.

For tray controls on Ubuntu, install gir1.2-ayatanaappindicator3-0.1. With a working indicator host, the panel stays hidden while you operate. Use the tray menu's Resume permission… to open the local confirmation panel. Losing the tray stops input and restores the panel. Details.

This is not an OS sandbox. Screenshots go to your MCP client, where its normal data handling applies. Keep sensitive apps closed during initial testing.

Troubleshooting and removal

Check dependencies and desktop readiness without starting control:

.venv/bin/linux-desktop-control doctor

To remove the client integration, stop any active task and run:

.venv/bin/linux-desktop-control uninstall

This removes the managed configuration, skill link, and guidance. Source files, dependencies, logs, and Stop state stay in place. To install without changing client configuration, use ./install without --register.

More details

• Reference: setup details, the seven tools, input limits, clipboard behavior, and live checks.

• Security and privacy: access boundaries and vulnerability reports.

• Contributing: development checks and bug reports.

• Verification and changelog.

• Optional accessibility companion: isolated, read-only AT-SPI observations alongside the existing input controller.

Native Wayland, remote access, separate background desktops, and accessibility element targeting are outside this beta's scope. It does not add Codex's native @Computer interface.