Skip to main content
Glama

manos

Hands for your AI agent on a real Linux desktop.

manos ("hands" in Spanish) is an MCP server that lets an agent such as Claude Code operate a KDE Plasma 6 desktop on Wayland: it browses and fills in websites, extracts data, works with spreadsheets and documents, and drives desktop apps. The model does the thinking. manos supplies the hands and the brakes, and it never sends, pays or types a secret without you.

you:    Look up tomorrow's forecast for Murcia on aemet.es, then put the LaLiga table in tabla.xlsx
        with a points-per-game column.
agent:  Max 26 °C, min 13 °C, 0 % rain. Heads-up: AEMET has a heavy-rain warning for the Vega del Segura.
        tabla.xlsx saved: 20 teams, column L = points per game. Verified by the server.

Why manos

Most computer-use agents look at screenshots and click on pixels. That is slow, expensive and fragile. manos takes the most reliable route available for each step, and drops to pixels only as a last resort:

Rung

How it acts

Used for

1

Application menu actions by name over D-Bus (KXmlGui)

Every KDE app: file_save, create_dir, edit_paste…

2

The accessibility tree (AT-SPI): roles, names, states

Any accessible app, GTK included

3

Screenshot plus synthetic keyboard and mouse (libei)

Whatever the first two cannot reach

Web

A dedicated Chromium over CDP, read as elements rather than pixels

Sites, forms, iframes, shadow DOM, dialogs

Office

Headless LibreOffice over UNO, no windows

.ods .xlsx .csv .odt .docx .pdf

In the benchmark, only 6.3 % of acting steps needed pixels.

Related MCP server: deskwright

Measured, not claimed

  • 35-task core benchmark, Claude Sonnet: 97 % pass (34/35), 0 false "done", $2.72 for the full run. The one miss was the agent stopping to ask which orders to notify, which is a correct stop.

  • Real-world round on a live desktop: 7/8 done, plus 1 correct handoff (a Cloudflare challenge), $1.20 in total. Every answer was checked against the real data.

  • The server checks "done" itself. submit_task_complete re-runs the task's acceptance criteria (files, cells, page state, windows) before it reports success. If it can't prove the task is done, it doesn't say so.

What it can do (30 tools)

  • Web. Open, navigate, observe the page as a numbered list of actionable elements, click, type, upload, answer native dialogs and work inside iframes.

  • Forms in one call. form_fill fills a whole form server-side: dates, input masks, custom selects, multi-step wizards and shadow DOM. It then re-reads every field and checks validity. Sending is a separate step that needs a one-time token.

  • Bulk extraction. Paginated tables or JSON APIs go straight to .csv/.ndjson without passing through the model's context. Extraction is resumable and backs off on 429 Retry-After.

  • Downloads. Saves a file or image by URL, or by pointing at a thumbnail or link on the page. It never overwrites an existing file.

  • Office without windows. Read, write (with formulas) and convert spreadsheets and documents.

  • Desktop apps. Launch apps, use their menus by name, read and act on any accessible UI, and place windows.

  • WebMCP. If a page declares its own tools, the agent calls them directly.

  • Gets cheaper with use.

    • A task the server has certified can be replayed step by step without re-thinking it.

    • After every certified task, manos records what worked in that app, and the next task reads it first.

  • Long tasks. Checkpoints, resume by name on another day, and ask_human for decisions only you can make.

Safety is the product

Brake

What it does

Emergency stop Meta+Alt+Esc

Freezes the browser and every action in ~10 ms through a cgroup freeze. It bypasses the MCP server entirely and can be undone.

Take the wheel Meta+Shift+B

Brings the agent's browser to you; the agent stays hands-off until you say "go on".

Handoff

Passwords, PINs, one-time codes, cards, captchas, anti-bot checks and logins go to you, never the model.

Two-step sends

Pay, send, delete and confirm need your approval, and replay never repeats them.

No secrets on disk

Your form profile (~/.config/manos/profile.toml) refuses to load if it looks like it contains a secret.

Blind spots by design

Password managers and other sensitive windows are never read. Hidden folders under ~ are never uploaded or opened.

Cookie banners

Only ever rejected, never accepted on your behalf.

Audit log

One line per action (tool, rung, ok/error) with no values, URLs or page text.

The agent's browser lives off-screen so it never covers your work. Ask the agent to work in watch mode and it stays visible, with shared control: you can solve a captcha right in the window while the agent keeps going.

Quick start

Requirements:

  • KDE Plasma 6 on Wayland, with a systemd user session

  • Python 3.14 and uv

  • LibreOffice with its Python UNO bridge, for the office tools

  • An MCP client; it is built and tuned for Claude Code

git clone https://github.com/Dgmtnz/manos && cd manos
uv sync
uv run playwright install chromium        # the agent's own browser, separate from yours
uv run --project . server.py --diag       # checks the machine: PASS / FAIL / SKIP per line

claude mcp add -s user manos -- uv run --project "$PWD" "$PWD/server.py"

panic/manos-panic --install               # then assign Meta+Alt+Esc in System Settings > Shortcuts
panic/manos-summon --install              # then assign Meta+Shift+B

Then just ask:

cd ~/Documents/somewhere && claude --model sonnet
> Read ~/Downloads/expenses.xlsx, total it by category and save the summary to summary.xlsx.
> In Dolphin, create Downloads/pdf-november and move this month's PDFs into it.
> Search Google Images for cats and download the 5 cutest to my Desktop. Let me watch.

The full user guide (in Spanish) is in docs/tutorial.md.

Testing

uv run --project . server.py --selftest                       # unit, browser and nested-KWin tests: "todo OK"
uv run --project . bench/run.py --core --model sonnet         # the 35-task benchmark

Desktop tests run in a nested, isolated KWin session, so they never touch your live desktop. The benchmark scores each task by an independent verifier, not by the agent's own claim.

Project layout

Path

What

server.py

The MCP server: tools, instructions, the completion gate

browser/

Chromium session, page observation, actions, forms, extraction, sanitising, consent, WebMCP

desktop/

AT-SPI, app menus, input over libei, screen capture, Spanish keymap

kwin.py

Window placement and the KWin scripts behind parking and the focus guard

office.py, office_worker.py

Headless LibreOffice

verify.py, replay.py, memory.py

Completion checks, certified replay, per-app memory

panic/

Emergency stop and takeover, independent of the server

skills/

Method and form-filling guides the agent loads on demand

bench/, tests/

Benchmark tasks with verifiers, and the self-test

Status

manos is a working, tested personal project, built and measured on one machine: CachyOS with Plasma 6.8 on Wayland. Its code comments, prompts and guide are in Spanish, and the prompts address the user as Diego. Change that in server.py if you adopt it. The browser path is pinned to the Playwright Chromium build in browser/session.py.

License

MIT

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      A
      maintenance
      Enables AI agents to automate Linux desktop GUI by launching and interacting with Wayland applications in isolated virtual KWin sessions, or connecting to live desktops for collaborative automation.
      159 PyPI
      81
      MIT
    • A
      license
      A
      quality
      B
      maintenance
      Enables AI agents to operate a real GNOME Wayland desktop through accessibility-tree widget actions, pointer/keyboard input, OCR, window management, and screen capture, optionally on a private headless session.
      33
      28 PyPI
      11
      Apache 2.0
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables an AI agent to see and control a Linux desktop via Wayland/Hyprland, providing screenshots, structured desktop state, pointer/keyboard input, semantic window/workspace tools, and a policy engine for safe execution.
      Apache 2.0
    • A
      license
      Not graded
      quality
      B
      maintenance
      Enables AI agents to automate and observe Linux desktop GUIs on KDE Plasma 6 Wayland through isolated virtual sessions or live desktop connections, providing 30 tools for input, clipboard, accessibility tree inspection, screenshots, and window management.
      2
      MIT