wayland-computer-use-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| WAYLAND_MCP_ACCESS_MODE | No | Access scope: 'window' (default), 'fullscreen', or 'both'. | window |
| WAYLAND_MCP_SOURCE_TYPE | No | Portal source type: '2' (window), '1' (fullscreen), or '3' (both). | |
| WAYLAND_MCP_DISPLAY_MODE | No | Display mode: 'live' (default) or 'virtual'. | live |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
Interactive Wayland window/desktop streaming, AT-SPI UI inspection, tree-first semantic navigation, reusable preset workflows, crash interception, and clamped input suite with reactive JSON UI delta feedback.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| logging | {} |
| prompts | {
"listChanged": true
} |
| resources | {
"subscribe": false,
"listChanged": true
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| launch_appC | Launches a Python GUI script or system app with auto-focus and portal session binding. If restart=True or if the app is already running, gracefully terminates the old process and launches a fresh instance. |
| restart_appB | Gracefully terminates and re-launches an active process launched by this MCP session, preserving its original launch command, arguments, and working directory. NOTE: Only applications launched by this MCP server can be restarted. External applications or user-opened windows cannot be restarted by the MCP server. |
| terminate_appB | Terminates an owned application process and closes idle virtual windows. |
| get_app_logsB | Retrieves captured stdout and stderr execution logs for an application process. |
| list_managed_appsA | Lists all active application processes managed by this MCP session. |
| inspect_ui_treeB | Inspects semantic accessibility tree and returns functional hierarchy + 1D interactive nodes. Includes compact functional component hierarchy, actionable elements, screen bounds, and process health by default. Set include_raw_tree=True only when debugging raw AT-SPI nodes. |
| interact_with_nodeB | Interacts with an interactive UI widget by 1D node ID (e.g. 'b1', 'e1') or label. Executes Level 1 AT-SPI actions silently (without moving the cursor), falling back to
Level 2 coordinate/keyboard input only when AT-SPI is unsupported, and returns the
post-action UI tree delta (modified/appeared widgets). Optional |
| batch_actionsA | Executes a sequence of consecutive UI interactions with fail-fast reporting. Supported step schemas:
|
| watch_ui_eventsB | Awaits UI settlement by monitoring real-time AT-SPI2 D-Bus mutation events. Returns captured mutations and whether the UI has settled, eliminating the need to re-query the full accessibility tree after triggers. |
| clickC | Dispatches a mouse click clamped within the active window boundaries. |
| double_clickC | Dispatches a double-click within the active window boundaries. |
| right_clickC | Dispatches a right-click within the active window boundaries. |
| hoverA | Positions cursor at coordinates without clicking to trigger hover states or tooltips. |
| dragC | Performs a clamped drag-and-drop gesture from start to end coordinates. |
| scrollB | Dispatches discrete mouse wheel scroll deltas (positive dy=down, negative dy=up). |
| type_textB | Injects keyboard characters. If coordinates (x, y) are provided, clicks first to focus. |
| key_combinationC | Dispatches allowed key combinations (e.g. ['ctrl', 'c']) after security filtering. |
| capture_window_frameB | Captures window frame as in-memory ImageContent (saves PNG when save_artifact=True). |
| clipboard_readA | Reads current text contents from the Wayland clipboard. |
| clipboard_writeC | Writes text into the Wayland clipboard. |
| window_controlA | Controls window state across any active Wayland compositor (KDE, GNOME, Hyprland, Sway). Supported actions:
|
| preset_workflowA | Manages and executes reusable multi-step interaction macro presets. Actions:
Step schema matches batch_actions:
|
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
| wayland_automation_guide | Returns system prompt extension and execution guidelines for Wayland GUI automation. |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
| get_system_prompt_resource | Direct resource link to the Wayland computer-use agent prompting guidelines. |
TDQS
Scored across 22 tools
Several tools overlap heavily: interact_with_node already supports click, double_click, right_click, hover, type, set_value, and clear, yet there are separate click/double_click/right_click/hover/type_text/key_combination tools, leaving the agent to guess whether to target a semantic node ID or raw coordinates. batch_actions and preset_workflow also both execute step sequences, differing only in persistence. Descriptions help (AT-SPI vs coordinate fallback), but boundaries remain fuzzy.
Everything is snake_case, but the conventions are mixed: verb_noun (launch_app, inspect_ui_tree, capture_window_frame), bare verbs (click, hover, drag, scroll), noun_verb (clipboard_read, clipboard_write), and noun phrases (window_control, batch_actions, preset_workflow). Readable but not a single predictable pattern.
At 22 tools the surface is on the heavy side for the domain, and much of that weight comes from the low-level mouse/keyboard primitives that duplicate what interact_with_node and batch_actions already do. The app-lifecycle and clipboard groups clearly earn their place, but the input layer could be consolidated.
Coverage is broad for a Wayland computer-use server: window state, app lifecycle with logs, accessibility tree inspection, semantic and raw input, batching, presets, UI event watching, frame capture, and clipboard read/write. Minor gaps exist around window enumeration/positioning and full-screen (non-window) capture, but core agent workflows are supported.