Skip to main content
Glama
BeckhamLabsLLC

Linux Desktop MCP Server

by BeckhamLabsLLC
OverviewSchemaRelated ServersScoreDiscussions
Python
Local

Linux Desktop MCP Server

PyPI version License: MIT Python 3.10+

Built with - This entire MCP server was developed using Claude Code, Anthropic's AI-powered coding assistant. We're proud to showcase what's possible with AI-assisted development!

An MCP server that provides Chrome-extension-level semantic element targeting for native Linux desktop applications using AT-SPI2 (Assistive Technology Service Provider Interface).

Features

  • Semantic Element References: Just like Chrome extension's ref_1, ref_2 system

  • Role Detection: Identifies buttons, text fields, links, menus, etc.

  • State Detection: Tracks focused, enabled, checked, editable states

  • Natural Language Search: Find elements by description ("save button", "search field")

  • Cross-Platform Input: Works on X11, Wayland, and XWayland

  • GTK/Qt/Electron Support: Works with any application that exposes accessibility

Installation

System Dependencies

# Ubuntu/Debian sudo apt install python3-pyatspi gir1.2-atspi-2.0 at-spi2-core # For X11 input simulation sudo apt install xdotool # For Wayland input simulation (recommended) # Install ydotool from source or your package manager # Then start the daemon: sudo ydotoold &

Python Package

# From PyPI pip install linux-desktop-mcp # Or from source git clone https://github.com/yourusername/linux-desktop-mcp.git cd linux-desktop-mcp pip install -e .

Enable Accessibility

Ensure accessibility is enabled in your desktop environment:

  • GNOME: Settings → Accessibility → Enable accessibility features

  • KDE: System Settings → Accessibility

  • Most modern desktops have this enabled by default

Configuration

Add to ~/.claude/settings.json:

{ "mcpServers": { "linux-desktop": { "command": "linux-desktop-mcp" } } }

Or if installed from source:

{ "mcpServers": { "linux-desktop": { "command": "python", "args": ["-m", "linux_desktop_mcp"] } } }

Available Tools

desktop_snapshot

Capture the accessibility tree with semantic element references.

Parameters: app_name: str (optional) - Filter to specific application max_depth: int (default: 15) - Tree traversal depth Returns: Tree of elements with ref_ids: - ref_1: [application] Firefox - ref_2: [frame] "GitHub - Mozilla Firefox" - ref_3: [button] "Back" (clickable) - ref_4: [entry] "Search or enter address" (editable, focused)

desktop_find

Find elements by natural language query.

Parameters: query: str - "save button", "search field", "menu containing File" app_name: str (optional) Returns: Matching elements with refs, states, and actions

desktop_click

Click an element by reference or coordinates.

Parameters: ref: str - Element reference (e.g., "ref_5") element: str - Human description for logging coordinate: [x, y] - Fallback if no ref button: left|right|middle click_type: single|double modifiers: [ctrl, shift, alt, super]

desktop_type

Type text into an element.

Parameters: text: str - Text to type ref: str - Element to focus first (optional) element: str - Human description clear_first: bool - Ctrl+A, Delete before typing submit: bool - Press Enter after

desktop_key

Press keyboard keys/shortcuts.

Parameters: key: str - Key name (Return, Tab, Escape, a, etc.) modifiers: [ctrl, shift, alt, super]

desktop_capabilities

Check available automation capabilities.

Example Usage

# 1. Take a snapshot to see available elements desktop_snapshot(app_name="Firefox") # 2. Find a specific element desktop_find(query="search bar") # 3. Click the element desktop_click(ref="ref_5", element="URL bar") # 4. Type into it desktop_type(text="https://github.com", ref="ref_5", submit=True) # 5. Use keyboard shortcuts desktop_key(key="t", modifiers=["ctrl"]) # New tab

Platform Support

Feature

X11

Wayland

XWayland

AT-SPI discovery

Full

Full

Full

Click by ref

Full

Full

Full

Type text

Full

Full

Full

ydotool input

Full

Full

Full

xdotool input

Full

No

Yes

Troubleshooting

"AT-SPI2 not available"

sudo apt install python3-pyatspi gir1.2-atspi-2.0 at-spi2-core

"AT-SPI2 registry not running"

Ensure accessibility is enabled in your desktop settings. You may need to log out and back in.

"No input backend available" (Wayland)

# Install and start ydotool daemon sudo ydotoold &

Elements not showing up

Some applications may not expose accessibility information. Modern GTK3/4, Qt5/6, and Electron apps generally work well.

Architecture

┌─────────────────────────────────────────────────────────────┐ │ MCP Protocol Layer │ │ (JSON-RPC over stdio, tool defs) │ └─────────────────────────────────────────────────────────────┘ │ ┌─────────────────────────────────────────────────────────────┐ │ Reference Manager │ │ (ref_1, ref_2 mapping, lifecycle, GC) │ └─────────────────────────────────────────────────────────────┘ │ ┌───────────────────┴───────────────────┐ │ │ ┌─────────────────────┐ ┌─────────────────────────┐ │ AT-SPI2 Backend │ │ Input Backends │ │ (pyatspi) │ │ (ydotool/xdotool/wtype) │ └─────────────────────┘ └─────────────────────────┘

Contributing

This project was created with Claude Code and we warmly welcome contributions! Whether you want to:

  • Report bugs or request features

  • Submit pull requests

  • Fork and build your own version

  • Improve documentation

We're very open to help and collaboration. See CONTRIBUTING.md for guidelines.

License

MIT - See LICENSE for details.

This server cannot be installed

-
security - not tested
A
license - permissive license
-
quality - not tested

How are these scores calculated?

Resources

New MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/BeckhamLabsLLC/linux-desktop-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server