use-computer-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| USE_COMPUTER_TESSERACT_LANGS | No | Additional OCR languages for tesseract, e.g. "eng osd srp srp_latn". |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| computerA | Mouse, keyboard and screen on the user's GNOME desktop. Coordinates are pixels in the most recent screenshot (it is downscaled, e.g. 1280x720 for a 1920x1080 screen; the tool maps them back). A screenshot waits briefly for the screen to stop changing. Text outside ASCII is pasted via the clipboard, which is restored afterwards. Actions other than screenshot/zoom return a short confirmation, not an image: take a screenshot when you need to see the outcome. |
| read_screenA | Accessibility tree of windows: one line per element with [ref], role, name, @(x,y,w,h) box in screenshot pixels, value, states and actions. Prefer refs over pixel guessing. Apps that do not expose accessibility (games, canvases, some Electron apps) show little or nothing; use screenshots and ocr there. 'positions=unresolved' means refs still work for actions but boxes are unknown. |
| findA | Find elements by name. Returns refs with boxes (accessibility) or, if nothing accessible matches, OCR text boxes with a click_at point in screenshot pixels. |
| form_inputA | Set a form control's value directly through accessibility, without typing. |
| ocrA | Read text from the screen with tesseract: lines with confidence and boxes in screenshot pixels. Use when accessibility has nothing (images, canvases, terminals, remote desktops). ~0.7 s for a full 1080p screen. |
| windowsA | List open windows (app, title, active, ref) from accessibility. |
| open_appA | Open or switch to an app through the Activities search (Super, type, Enter). Confirm with a screenshot afterwards. |
| wait_forB | Wait for something instead of sleeping. Returns met=true/false. |
| clipboardB | Read or replace the clipboard text. |
| computer_batchA | Run several steps in one round trip, e.g. click a field, type, press Return, wait, screenshot. Returns every step's text and any images, in order. |
| listenA | Record audio for a fixed time and transcribe it with Whisper (local, CPU). The recording is deleted afterwards. Start playback before calling. Transcription takes about as long as the audio (minimum ~15 s, Whisper works in 30-second windows). |
| transcribeB | Transcribe an audio/video file with Whisper (local, CPU). |
| sessionB | Inspect or end the control session. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 13 tools
Most tools have distinct roles: read_screen, find, and ocr separate accessibility-tree reading from element lookup from text extraction, while listen and transcribe split recording from file transcription. The main ambiguity is between computer and computer_batch, but the batch tool is clearly described as a multi-step version of the same action set.
Names are readable and consistently lowercase snake_case, but conventions are mixed: some are imperative verb phrases (read_screen, open_app, wait_for, form_input) while others are bare nouns (computer, ocr, windows, clipboard, session) or awkward compounds (computer_batch). This is mildly inconsistent but not chaotic.
13 tools is well-scoped for a desktop-control server; input, screen perception, accessibility, OCR, window management, waiting, clipboard, batch execution, audio transcription, and session control each earn a place. There is no obvious bloat or redundancy at the tool-count level.
The surface covers core desktop-automation workflows well: mouse/keyboard input, screenshots, accessibility trees, OCR, app/window management, form filling, waiting, clipboard, and session control. Minor gaps exist, such as no dedicated audio-output tool and relying on computer for primitive actions like screenshot or typing, but these are workable.