Skip to main content
Glama
5omeOtherGuy

linux_desktop

by 5omeOtherGuy

Linux Desktop for Codex

An independent, MIT-licensed X11 desktop MCP server with persistent Python execution. Built and tested with Codex; usable by compatible local MCP hosts. Not affiliated with OpenAI.

A persistent, local MCP capability for the logged-in Linux desktop. Codex receives screenshots and accessible controls, then operates real applications through AT-SPI and X11 mouse/keyboard input. Five discoverable tools provide status, observation, Python execution, cancellation and reset. No screenshots need to be copied through temporary files during ordinary use.

Experimental 0.1.0. Validated on a local X11/XFCE session. The isolated Xvfb/Openbox test run has unresolved failures; it is not a supported headless configuration. Native Wayland, locked sessions, remote machines and a takeover overlay are not implemented. Applications without accessibility can be operated visually with physical input; compatibility with every app is not guaranteed.

Why this exists

The useful distinction is the combination of five tools, a persistent Python namespace, real AT-SPI controls, and recoverable physical input. An agent can retain a window reference, loop over actions, wait for an app state, and return a screenshot in one call. X11 connections stay open; accessibility observations can return only changed nodes. No app repackaging, root input daemon, or vendor binary is required.

There are good alternatives, especially for Wayland. This project favors a small, inspectable X11 integration with local Codex acceptance evidence. Persistent Python processes consume memory, and a smaller tool list does not by itself prove lower token cost. There is no measured claim of better speed or resource usage. See the comparison and tradeoffs before choosing.

Dedicated test app after a fresh Codex session entered text, checked a box and saved

Related MCP server: mcp-linux-desktop

Install and use

git clone https://github.com/5omeOtherGuy/linux-computer-use.git
cd linux-computer-use
./scripts/install.py --register

Requires system Python 3.12+ and uv. On Ubuntu 24.04, install desktop dependencies with:

sudo apt install python3-gi python3-pyatspi python3-pil gir1.2-atspi-2.0

To explicitly authorize this server's tools without per-call prompts, run ./scripts/install.py --register --allow-desktop-control instead. To install for another MCP host, omit --register, then configure its stdio command as the absolute path to scripts/launch.

The installer uses system Python 3.12+, uv, and a private virtual environment with locked Python dependencies. System GI, pyatspi, Pillow, AT-SPI and an X11 server with XTEST are required. The launcher recovers missing graphical-session environment variables from the user's systemd manager. It does not guess a display or store credentials. The server is stdio-only, runs as the current user and starts its desktop worker lazily; no root daemon or listening network port is installed.

Start a new Codex session after registration. Ask, for example: “Use linux_desktop to open Calculator and calculate 17 × 23.” MCP initialization supplies the API instructions, so this capability does not depend on adding a skill or modifying the skills catalog. Existing sessions may retain their original tool catalog.

--allow-desktop-control implements the owner's authorization by setting default_tools_approval_mode = "approve" for this server alone. Without it, Codex can require approval for desktop actions; a noninteractive session configured with approval_policy = "never" then rejects those actions. The installer uses Codex's versioned config API and does not change other servers, skills, or global approval settings. Omit that flag when per-call approval is desired.

Tests additionally use gir1.2-gtk-3.0 and python3-pytest. Accessibility must be enabled in the graphical session. On desktops honoring the GNOME setting, gsettings set org.gnome.desktop.interface toolkit-accessibility true enables it; other desktops may require their accessibility settings. App support varies, and some apps need their own accessibility setting or a restart. Keep this checkout at its installed path; rerun registration after intentionally removing an old entry if relocating it.

To unregister without changing apps or their data:

codex mcp remove linux_desktop

Tools and persistent API

desktop_status discovers capabilities, monitors and native window IDs. desktop_observe returns accessibility state and an image together. Window screenshots focus that window; a full desktop observation leaves focus alone. desktop_exec runs synchronous Python in a persistent worker, with desktop and display preloaded. desktop_cancel stops that worker independently, and desktop_reset discards its variables. Both release owned input and leave applications open.

Example desktop_exec code:

print(desktop.apps())                    # exact installed desktop-file IDs
desktop.launch("org.gnome.Calculator.desktop")
print(desktop.windows())                 # choose a unique title or window ID
calc = desktop.window("Calculator")
calc.observe()                          # emits state and screenshot automatically
for name in ["1", "7", "×", "2", "3", "="]:
    calc.find(name=name, role="push button").click()
calc.wait_for(name="editbar", text="391")
calc.observe()

Observe actual labels before acting; the names above match the tested Calculator, not every calculator. Variables such as calc survive subsequent calls until reset or cancellation.

Operation

API

Observe

w.observe(screenshot=True, max_elements=250, diff=True)

Locate

w.find(name=..., role=..., text=...), w.element(observed_id)

Semantic input

e.click(action=None), e.set_value(value), e.select_text(unique_text)

Wait for state

w.wait_for(name=..., role=..., text=..., checked=..., timeout=5)

Pointer

w.click(x,y,button=1,count=1), w.drag([[x,y],...],duration=.25)

Keyboard

w.press_key("Ctrl+a"), w.type_text("Grüße 漢字 🙂")

Scroll

w.scroll("down",steps=3,x=...,y=...)

Window

w.info(), w.focus(), w.configure(x=...,y=...,width=...,height=...), w.close()

Whole desktop

desktop.observe(), then desktop.move/click/press_key/type_text/scroll/drag(...)

Image

display(desktop.capture()), display(desktop.capture([x,y,width,height]))

Window input uses client-area coordinates and focuses the chosen window. Whole-desktop input uses desktop pixels and retains current focus, covering panels, window decorations, menus and other OS surfaces. Image metadata gives the crop origin and scale. Physical input requires a preceding observation; moving/resizing a window invalidates its coordinates. Never infer that a successful event means the intended result occurred: inspect or wait for the resulting state.

Accessibility observations use stable element IDs and return changes on subsequent observations. Request diff=False to recover a full tree. Searches require an unambiguous, complete visible tree; large/truncated trees require an explicit observed handle or visual input. Handles are checked against their original app window and current ancestry before use.

Runtime behavior

The MCP server and desktop worker have separate lifetimes. A worker keeps its X11 connection, accessibility event listener and Python variables between calls. Batch actions and local state checks in one call to avoid model round trips. Screenshot capture stays in memory. Event-assisted waits avoid fixed long delays; compact accessibility diffs reduce repeat output. These reduce overhead but do not eliminate model latency or application response time. No comparative performance claim is made without measurements.

The persistent XTEST path handles input. If a separately installed OpenAI @oai/sky Linux binary is detected, an optional desktop.native("move",x=...,y=...) pointer route is available; other input stays on the tracked driver so cancellation can release it. No vendor binary is patched, copied, or distributed. Sky is not a dependency and may change after ChatGPT updates.

A cross-client lease prevents simultaneous automation batches on the same display. Owned keys, buttons and temporary Unicode key mappings are recorded before use. Normal completion, failure, timeout and cancellation release them; an independent recovery process handles a terminated worker. The default execution deadline is 30 seconds, configurable up to 60 seconds. Output is bounded. After interruption, observe the current app before retrying because earlier actions may already have happened.

Python execution has the user's OS privileges; it is not an OS sandbox. Tool results and visible app content do not provide permission for unrelated actions. This implementation coordinates its own clients, not concurrent human input or unrelated automation. Current clipboard contents are preserved by text entry. Complex input methods, unusual layouts, secure prompts, mixed-DPI/multiple-monitor setups and non-X11 sessions need separate validation. Read-only annotations are not used for observations that focus windows.

Validation and reference material

.venv/bin/python -m pytest -q

The tests manipulate a dedicated visible GTK app, exercise real mouse/keyboard and AT-SPI behavior, use a native file chooser, and verify results independently through fixture state. They cover Unicode, menus, dragging, scrolling, stale/ambiguous/disabled targets, geometry changes, state diffs, persistent MCP state, screenshots, cancellation, timeouts and competing clients. Run them when brief desktop focus changes are acceptable.

See ACCEPTANCE.md for the actual installed-session results and RESEARCH.md for the implementation boundary and public sources.

See CONTRIBUTING.md for the isolated test recipe and SECURITY.md for the execution boundary. The MIT license covers this project's source and documentation; dependencies and separately installed vendor software retain their own licenses.

Available Tools

5 tools
desktop_cancelA

Stop the running desktop worker independently, release its held input, and reset persistent variables.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations only indicate readOnlyHint=false, so the description carries the burden of disclosing side effects. It does this well by explicitly naming three behaviors: stopping the worker, releasing held input, and resetting persistent variables. It does not cover error cases or reversibility, but the main mutating impact is transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with the primary action front-loaded and every clause adding real information: stopping the worker, releasing input, and resetting variables. There is no filler or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, the description provides the essential contract: what is stopped, what is released, and what state is reset. It could be more complete by mentioning behavior when no worker is running or what success/error output looks like, but the core calling context is adequately covered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema description coverage is 100%, so there are no parameter semantics to document. The description appropriately adds no parameter-related noise, earning the baseline for a no-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Stop'), a clear target ('the running desktop worker'), and lists the concrete side effects ('release its held input', 'reset persistent variables'). This clearly distinguishes it from sibling tools like desktop_status, desktop_observe, and desktop_exec.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Stop the running desktop worker' implies the tool should be used when a worker is currently running and needs to be halted. However, it does not explicitly compare against desktop_reset or state when not to use this tool, so the usage guidance remains implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

desktop_execB
Destructive

Run persistent Python desktop automation. desktop and display(image) are preloaded. Batch actions, verify the result, and call w.observe() to return UI evidence. See server instructions for the API.

ParametersJSON Schema
NameRequiredDescriptionDefault
codeYes
timeoutNo

TDQS

B3.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds meaningful operational context beyond the annotations: the environment is persistent across calls, desktop and display(image) are preloaded, and w.observe() is needed to obtain UI evidence. This is useful behavior disclosure that is not present in the schema or annotations. It aligns with the destructiveHint=true annotation by implying the execution can change the desktop state, and there is no contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise at three sentences, front-loads the core purpose, and avoids filler. Each sentence adds useful information, though the final 'See server instructions' clause is a deferral rather than self-contained content. Overall, it is efficiently structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is complex (persistent Python desktop automation) yet has no output schema, no parameter descriptions, and minimal annotation depth beyond a destructive hint. The description leaves key gaps: return value behavior, how to structure code, what 'persistent' means across invocations, and how the preloaded objects work. 'See server instructions' shifts essential invocation details elsewhere, making the definition incomplete for an agent to use reliably.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not directly explain the 'code' or 'timeout' parameters. It hints that code should use preloaded objects and batch actions, but it does not specify expected syntax, output, or timeout behavior. The description fails to compensate for the complete lack of parameter documentation in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with 'Run persistent Python desktop automation,' a specific verb and resource that clearly establishes this as the execution entrypoint among its siblings (status, observe, cancel, reset). It goes beyond a tautology and gives the agent a concrete action. It could be slightly more explicit about what desktop automation encompasses, but it is distinguishable from the related tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description offers usage guidance such as 'Batch actions, verify the result, and call w.observe() to return UI evidence,' which implies a workflow. However, it does not explicitly state when to choose desktop_exec over desktop_observe, desktop_status, or the other siblings, nor does it discuss limitations or alternatives. The reference to server instructions pushes important decision-making outside the definition.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

desktop_observeC

Observe controls and screenshot together; a window screenshot focuses that window. Omit window for the full desktop.

ParametersJSON Schema
NameRequiredDescriptionDefault
diffNo
windowNo
screenshotNo
max_elementsNo

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description adds a useful behavioral detail beyond the annotations: 'a window screenshot focuses that window,' which is a potential side effect. With readOnlyHint=false, this is meaningful context. However, it does not disclose whether observing controls has other side effects, what diff does, or what the output looks like, leaving some behavioral ambiguity.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise, with two front-loaded sentences and no filler. It earns its space by stating the core behavior and a key side effect. It could be slightly longer to cover parameters, but as written it is efficiently structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 4 parameters, 0% schema coverage, and no output schema, the description is too sparse to fully prepare an agent for correct invocation. It lacks explanations for diff, screenshot, and max_elements, and does not clarify how this tool relates to desktop_status. Significant gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It partially explains the window parameter by noting that omitting it captures the full desktop. But it says nothing about diff, screenshot, or max_elements, all of which are undocumented in both the schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Observe controls and screenshot together') and the resource being acted on. It also distinguishes between window and full-desktop modes, which gives concrete scope. However, it does not explicitly differentiate itself from sibling tools like desktop_status, so the differentiation is not fully explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus desktop_status, desktop_exec, desktop_cancel, or desktop_reset. It only gives parameter-level guidance ('Omit window for the full desktop'), which helps with invocation but not tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

desktop_resetA

Reset the persistent desktop runtime and release input. Does not close user applications.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint=false annotation already indicates a mutating operation. The description adds useful behavioral context by stating that the runtime is reset and input is released, while reassuring that user applications remain open. It does not detail exactly what state is lost, but it goes beyond the annotation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with the core action front-loaded and the safety boundary in a concise follow-up. Every word earns its place; there is no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, the description adequately conveys purpose and a critical safety guarantee. It could be more explicit about what persistent runtime state is lost or whether the reset is reversible, but the core information needed to invoke the tool is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and an empty schema, so there is no parameter meaning for the description to add. The baseline of 4 is appropriate because no parameter documentation is needed or expected.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific action 'Reset' and the resource 'persistent desktop runtime,' adding 'release input' to capture the secondary effect. It also explicitly states what it does not do ('Does not close user applications'), which distinguishes it from sibling tools like desktop_exec or desktop_cancel.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for when the tool is appropriate: resetting the persistent runtime and releasing input. It does not explicitly reference alternatives or exclusions, but the simple, well-defined action is enough for an agent to select it appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

desktop_statusA
Read-only

Discover Linux desktop capabilities, screen geometry and windows before choosing a target.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The readOnlyHint annotation already establishes that this is a safe, non-mutating operation. The description adds value by specifying what gets inspected (capabilities, screen geometry, windows), which is useful behavioral context beyond the annotation alone.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence conveys purpose, scope, and usage timing with no wasted words. Every phrase earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless read-only status tool, the description is complete enough: it states what the tool reveals and when to invoke it. It does not describe the output format, but given the absence of an output schema and the simple status nature, this is a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero parameters, so there is nothing for the description to explain about arguments. This matches the baseline for a parameterless tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states that this tool discovers Linux desktop capabilities, screen geometry, and windows, which is a specific verb and resource. It does not explicitly distinguish itself from sibling desktop_observe, but the naming and content make its role reasonably clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'before choosing a target' gives a clear usage context: this tool is intended as a pre-action discovery step. It does not mention when not to use it or name alternative tools, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 5 tool updatesv0.1.0
    • First observeddesktop_cancel
    • First observeddesktop_exec
    • First observeddesktop_observe
    • First observeddesktop_reset
    • First observeddesktop_status

TDQS

A3.5/5.0
Disambiguation2/5

desktop_status, desktop_observe, and desktop_exec are largely distinct, but desktop_cancel and desktop_reset have heavily overlapping descriptions—both release input and reset persistent variables. Additionally, desktop_exec's ability to call w.observe() blurs the boundary with the standalone desktop_observe tool.

Naming Consistency4/5

All tools share the consistent desktop_ prefix and use snake_case, making the naming predictable. The pattern is mostly desktop_<action>, though 'status' is a noun rather than a verb and 'exec' is an abbreviation, which are minor deviations.

Tool Count5/5

Five tools is a well-scoped size for a desktop automation server. Each tool covers a phase of the workflow—discover, observe, execute, cancel, and reset—without unnecessary bloat.

Completeness4/5

The set covers the core lifecycle of discovery, observation, execution, and cleanup, and desktop_exec's arbitrary Python capability provides broad automation coverage. A notable minor gap is the lack of a clear way to query the runtime/worker state, which would help agents decide between cancel and reset.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Controls a real Linux desktop from any MCP host. Reads accessibility trees, takes screenshots, and drives clicks, scrolls, and keystrokes across GNOME, KDE/KWin, Hyprland, i3, and COSMIC.
    528
    487
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables full Linux desktop control including windows, mouse, keyboard, clipboard, audio, screenshots, OCR, accessibility, and system management through MCP-compatible AI agents.
    1
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables MCP clients to control a Linux/X11 desktop like a human: see the screen, move the mouse, click UI elements via the accessibility tree, type text, and manage windows.
    MIT

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/5omeOtherGuy/linux-computer-use'

If you have feedback or need assistance with the MCP directory API, please join our Discord server