Skip to main content
Glama

perch

A small MCP server for desktop control. It lets an AI agent see and use the apps on your desktop with five compact tools, and keeps what it adds to the agent's context small.

perch does no desktop work itself. On Linux it runs computer-use-linux, which handles accessibility trees, screenshots, Wayland and X11 input, and window targeting on GNOME, KDE, Hyprland, Sway and others. perch shows the agent a shorter interface over it. Windows and macOS are planned.

Tools

Tool

Does

read_windows

Open windows: id, app, title, focus, workspace

read_window

A window's buttons, fields and text as compact lines: [e34 button "Save" disabled]

read_screenshot

A picture of one window (or the screen), or a PNG file

read_status

Whether accessibility, input and the window list work, and what blocks them

act_window

Steps in one window: click, type, key, set, action, scroll, drag, focus, wait

What it does for the agent:

  • e-refs. read_window lists elements as e34; act_window turns them into the server's element index and snapshot.

  • Screenshot pixels. [x, y] in act_window are pixels of the window's last read_screenshot; perch converts them.

  • Your screen stays yours where it can. Clicking an element whose primary action is click, press or toggle runs that action without focusing the window, on every desktop. On Hyprland, keys and window screenshots also stay in the background, so a test app on a hidden workspace never comes to the front; elsewhere both bring the window forward. Other elements (a table cell that expands or edits) are refused with their action names, to use as an action step. Mouse steps need an explicit focus step, and keys for the focused window are refused if you moved the focus.

  • Reused window ids. If a window closed and another app got its id, act_window refuses instead of acting in the wrong app.

  • Typed text is never echoed in results.

Size, measured against computer-use-linux 0.7.53 on Thunar:

computer-use-linux

perch

Tool list

19 tools, 67.8 KB

5 tools, 6.1 KB

Server instructions

7.6 KB

none

One window's tree

31.5 KB

1.1 KB

Related MCP server: planchette

Use

{
  "mcpServers": {
    "perch": { "command": "npx", "args": ["-y", "@redfoxxo/perch@1"] }
  }
}

Node.js 22.12 or newer. On Linux, computer-use-linux comes with perch (its binary downloads on first use) and needs the setup its README describes: accessibility on, and an input backend.

Variable

Default

Effect

COMPUTER_USE_LINUX_HYPRLAND_BACKGROUND

1

Hyprland: keys and window screenshots without focusing the window. 0 turns it off

Other COMPUTER_USE_LINUX_* variables pass through unchanged.

On GNOME Wayland, computer-use-linux lists and focuses windows through a small Shell extension. Install it with the version perch runs, then log out and back in:

npx -y @redfoxxo/perch@1 setup-window-targeting

Develop

npm ci
npm test
npm run typecheck
npm run build   # build/index.js is the server

License

MIT

Available Tools

5 tools
act_windowA
Destructive

Run steps in one window, in order, stopping at the first that fails: click, type, key, set, action, scroll, drag, focus, wait. Targets are e-refs from read_window or [x, y] pixels of its read_screenshot; drag takes pixels of a whole-screen read_screenshot. Element clicks, set and action never bring the window to the front; keys and typing don't on Hyprland, elsewhere they focus it. Mouse steps ([x, y] clicks, right or double clicks, scroll, drag) need a focus step first, which shows the window to the user. focused: true sends keys to the focused window, refused if the focus moved. then: tree (default after read_window) or screenshot (brings the window forward except on Hyprland) shows the result.

Inputs: steps (required), window (required), then.

read_screenshotA

Picture of one window (or the whole screen without window), longest side max_edge px. A window is captured in the background on Hyprland; elsewhere it comes to the front. Costs image tokens: prefer read_window. With file, saves a PNG there instead.

Inputs: file, window, max_edge.

read_statusA
Read-only

Whether desktop control works here: accessibility, input, windows, and what blocks it.

Inputs: no named parameters.

read_windowA
Read-only

A window's buttons, fields and text as compact lines, e.g. [e34 button "Save" disabled]. Act on them with act_window using the e-refs; a new read_window (of any window) replaces them.

Inputs: window (required).

read_windowsA
Read-only

Open windows, one per line: id, app, title, focused, workspace.

Inputs: no named parameters.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4/5.0

    Scored across 5 tools

    Disambiguation4/5

    read_window (element tree of one window) vs read_screenshot (image) is cleanly separated, and act_window is clearly the write-side counterpoint. The only real risk is read_window vs read_windows, where the plural name could be misread as 'read contents of all windows' rather than 'list open windows', though the description disambiguates.

    Naming Consistency5/5

    All names are snake_case verb_noun (read_window, read_windows, read_screenshot, read_status, act_window), with a natural read/act split that maps to inspect vs. manipulate. No mixed conventions or vague verbs like process/run.

    Tool Count5/5

    Five tools is tight and well-scoped: four read-side probes (element tree, window list, screenshot, capability status) plus one consolidated act_window that absorbs click/type/key/scroll/drag/focus/wait as steps. Nothing feels redundant or absent on count grounds.

    Completeness4/5

    The lifecycle of inspect-then-act on existing windows is fully covered, including a preflight read_status and post-action 'then' verification. Gaps remain around launching apps/closing windows and clipboard access, so an agent can inspect and drive open windows but cannot start a program itself.

    Maintenance

    ActivityMaintained
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables comprehensive Windows desktop automation including screen capture, OCR text extraction, mouse/keyboard control, window management, process control, and clipboard operations through 25+ tools for AI agents.
      38 PyPI
      8
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables coding agents to control other application windows by providing primitives for clicking, typing, capturing screenshots, and window management. Supports macOS, Windows, and Linux.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Lets AI agents see and control desktop applications through the accessibility layer, enabling clicking, typing, scrolling, dragging, and window/app management across macOS, Windows, and Linux entirely on the local machine.
      25
      MIT