Skip to main content
Glama

perch

A small MCP server for desktop control. It lets an AI agent see and use the apps on your desktop with five compact tools, and keeps what it adds to the agent's context small.

perch does no desktop work itself. On Linux it runs computer-use-linux (accessibility trees, screenshots, Wayland and X11 input, window targeting on GNOME, KDE, Hyprland, Sway and others). On Windows it runs the Windows runtime of open-computer-use (UI Automation, window messages, window capture). Both come with perch, behind the same short interface. macOS is not supported yet.

Requirements

  • Node.js 22.12 or newer.

  • Linux: x64 or arm64 with glibc (the builds computer-use-linux ships), in a desktop session. Accessibility on, and an input backend, as computer-use-linux's quick start describes. Its binary downloads on first use if npm skipped its install script.

  • GNOME Wayland: computer-use-linux lists and focuses windows through a small Shell extension. Install it with the version perch runs, then log out and back in:

    npx -y @redfoxxo/perch@1 setup-window-targeting
  • Windows: nothing else to install. Run perch in the signed-in desktop session, not over SSH or as a service.

read_status tells what is missing on a desktop, and how to fix it.

Related MCP server: planchette

Setup

perch runs over stdio; npx -y @redfoxxo/perch@1 starts it.

Claude Code:

claude mcp add perch -- npx -y @redfoxxo/perch@1

On native Windows, start npx through cmd /c: claude mcp add perch -- cmd /c npx -y @redfoxxo/perch@1.

opencode (opencode.json):

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "perch": { "type": "local", "command": ["npx", "-y", "@redfoxxo/perch@1"] }
  }
}

Other clients (mcpServers JSON):

{
  "mcpServers": {
    "perch": { "command": "npx", "args": ["-y", "@redfoxxo/perch@1"] }
  }
}

Environment variables (Linux):

Variable

Default

Effect

COMPUTER_USE_LINUX_HYPRLAND_BACKGROUND

1

Hyprland: keys and window screenshots without focusing the window. 0 turns it off

COMPUTER_USE_LINUX_AGENT_NAME

the client's name

Name computer-use-linux shows on screen while it acts

Other COMPUTER_USE_LINUX_* variables pass through to computer-use-linux unchanged. On Windows, perch starts the runtime with __COMPAT_LAYER=HighDpiAware unless you set that variable, so window positions and UI Automation coordinates are both in pixels on scaled displays.

Tools

Tool

Does

read_windows

Open windows: id, app, title, focus, workspace

read_window

A window's buttons, fields and text as compact lines: [e34 button "Save" disabled]

read_screenshot

A picture of one window (or the screen), or a new PNG file

read_status

Whether accessibility, input and the window list work, and what blocks them

act_window

Steps in one window: click, type, key, set, action, scroll, drag, focus, wait

What perch does for the agent:

  • e-refs. read_window lists elements as e34; act_window turns them into the server's element index and snapshot. A new read_window replaces them.

  • Screenshot pixels. [x, y] in act_window are pixels of the window's last read_screenshot; perch converts them, and refuses points outside that image.

  • Your screen stays yours where it can. Clicking an element whose primary action is click, press or toggle (Windows: invoke, select or toggle) runs that action without focusing the window. Other elements (a table cell that expands or edits) are refused with their action names, to use as an action step. On Hyprland and Windows, keys, typing and window screenshots stay in the background too; elsewhere they bring the window forward. Mouse steps need an explicit focus step, and keys for the focused window are refused if the focus moved.

  • Reused window ids. If a window closed and another app got its id, act_window refuses instead of acting in the wrong app.

  • Typed text is never echoed in results. Long text and field values in trees are cut at 1,000 characters.

Size, measured against computer-use-linux 0.7.53 on Thunar:

computer-use-linux

perch

Tool list

19 tools, 67.8 KB

5 tools, 4.8 KB

Server instructions

7.6 KB

none

One window's tree

31.5 KB

1.1 KB

Limitations

  • Apps without accessibility (old toolkits, Wine, games, canvases) show no tree; use read_screenshot and [x, y] clicks there.

  • Drags on Linux take pixels of a whole-screen screenshot; on Windows, pixels of the window's screenshot, like clicks.

  • Windows, background input: keys and typing are posted to the window as messages. Classic Win32 fields take them; some newer apps (Chromium, Electron, WinUI) ignore them, so the step says to check the result. focused: true after a focus step sends real keystrokes instead.

  • Windows, classic controls come forward when UI Automation acts on them: in CI, a WinForms button and check box (click, action) and Notepad's text field (set) all brought their window to the front. The step then says "the app came to the front".

  • Windows, trees show no disabled or checked state: the runtime doesn't report them.

  • Windows, refused apps: open-computer-use won't touch terminals, PowerShell, password managers or Windows Security. Apps running as administrator can be read but not clicked or typed into unless perch runs as administrator too.

  • Windows, speed: the runtime runs every operation in a new process, so a crashing capture can't take perch down. On GitHub's Windows runner, reads took 0.1 s and act_window calls 0.5–1.2 s.

  • Install size: npm installs both runtimes (about 40 MB) on every platform, since open-computer-use's package doesn't declare its platforms.

Development

See DEVELOPMENT.md.

License

MIT

Available Tools

5 tools
act_windowA
Destructive

Run steps in one window, in order; stops at the first that fails. Targets: e-refs from read_window, or [x, y] pixels of this window's last read_screenshot (drag: of a whole-screen read_screenshot; Windows: of the window's). Element clicks, set and action don't bring the window to the front (Windows: classic controls may, and the step says so). Keys and typing stay in the background on Hyprland and Windows (some Windows apps ignore them: check); elsewhere they focus the window. Mouse steps ([x, y] clicks, right or double clicks, scroll, drag) need a focus step first, which shows the window to the user. focused: true sends real keys (any layout and characters) to the window after a focus step; refused if the focus moved. then: tree (default after read_window), screenshot (raises the window except on Hyprland and Windows) or none.

Inputs: steps (required), window (required), then.

read_screenshotA

Picture of one window (or the whole screen without window), longest side max_edge px. A window is captured in the background on Hyprland and Windows; elsewhere it comes to the front. Costs image tokens: prefer read_window. With file, saves a PNG there instead.

Inputs: file, window, max_edge.

read_statusA
Read-only

Whether desktop control works here: accessibility, input, windows, and what blocks it.

Inputs: no named parameters.

read_windowA
Read-only

A window's buttons, fields and text as compact lines, e.g. [e34 button "Save" disabled]. Act on them with act_window using the e-refs; a new read_window (of any window) replaces them.

Inputs: window (required).

read_windowsB
Read-only

Open windows, one per line: id, app, title, focused, workspace.

Inputs: no named parameters.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A3.9/5.0

    Scored across 5 tools

    Disambiguation4/5

    read_windows (enumerate all windows) and read_window (inspect one window's element tree) are the only pair at risk of confusion, and their descriptions make the plural/singular distinction workable. read_screenshot, read_status, and act_window each target clearly different things: pixels, capability diagnostics, and mutation.

    Naming Consistency5/5

    All five names follow a strict verb_noun pattern, with read_* for observation tools and act_window for the sole mutating tool. The read/act split is semantically meaningful rather than an arbitrary deviation.

    Tool Count5/5

    Five tools is tightly scoped to the domain: enumerate, inspect, screenshot, check capability, and act. Each tool occupies a necessary slot in the perceive-then-act loop, with no redundant or filler entries.

    Completeness4/5

    The read-and-act lifecycle for existing windows is fully covered, including status checks and image fallback. Minor gaps remain: there is no way to launch an application or explicitly close a window outside of clicking an in-app control, which an agent may need for common desktop workflows.

    Maintenance

    ActivityMaintained
    ResponsivenessOutdated data

    Related MCP Connectors

    Related MCP Servers

    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables comprehensive Windows desktop automation including screen capture, OCR text extraction, mouse/keyboard control, window management, process control, and clipboard operations through 25+ tools for AI agents.
      55 PyPI
      8
      MIT
    • A
      license
      Not graded
      quality
      C
      maintenance
      Enables coding agents to control other application windows by providing primitives for clicking, typing, capturing screenshots, and window management. Supports macOS, Windows, and Linux.
      MIT
    • A
      license
      Not graded
      quality
      B
      maintenance
      Lets AI agents see and control desktop applications through the accessibility layer, enabling clicking, typing, scrolling, dragging, and window/app management across macOS, Windows, and Linux entirely on the local machine.
      27
      MIT