desktop-mcp
desktop-mcp
Visión y control de escritorio para LLMs en Linux, incluido Wayland — el modelo toma una captura de pantalla de toda la pantalla y hace clic, escribe y desplaza sobre lo que ve. MCP simple sobre stdio, por lo que funciona en Claude Code, Claude Desktop, Cursor o cualquier otro cliente MCP.
Léelo primero: puede que no lo necesites
Anthropic incluye el uso de computadora de forma nativa, y donde eso funcione deberías usarlo en su lugar: está mejor integrado y realmente se mantiene:
macOS y Windows: Claude Desktop tiene el uso de computadora integrado, en Cowork y Claude Code. Controla tu escritorio real. (docs)
Construyendo tu propio agente: la API de Claude tiene una herramienta
computerde primera parte (GA como el conjunto de herramientascomputer_toolset_20260801). (docs)
El vacío que esto llena es Linux. A partir de agosto de 2026, el uso de computadora no está disponible en la aplicación de escritorio de Linux, y la automatización de escritorio en Wayland es genuinamente incómoda de una manera que X11 nunca lo fue:
El compositor de COSMIC no implementa
wlr-screencopy, por lo quegrimfalla por completo y cada herramienta de captura de pantalla necesita una ruta por compositor.No hay protocolo para consultar la posición del cursor, ni para enumerar las ventanas de otro cliente.
La entrada debe ir por debajo del compositor a través de
/dev/uinput, porque elydotoolcomúnmente empaquetado expone un puntero relativo cuyos saltos largos se estropean con la aceleración del puntero.
Si estás en macOS o Windows, este repositorio es en su mayoría redundante. Si estás en Linux — particularmente Wayland — es la opción que funciona.
Los servidores MCP de automatización de escritorio no son una categoría nueva; hay miles de servidores MCP. Lo específico aquí es la ruta de entrada absoluta-uinput nativa de Wayland, medida con precisión de píxel en COSMIC.
Related MCP server: mcp-linux-desktop
Estado
Escrito para ser multiplataforma, pero solo una configuración se ha ejercitado realmente. El resto son rutas de código, no evidencia: trátalo en consecuencia.
Plataforma | Captura de pantalla | Entrada | Probado |
Linux / Wayland |
|
| Sí — solo COSMIC |
Linux / X11 |
|
| No |
macOS |
|
| No |
Windows |
|
| No |
Qué se probó y cómo
Verificado de extremo a extremo en:
Pop!_OS 24.04 LTS, kernel 7.0.11, COSMIC (
cosmic-comp1.0.0), Wayland, pantalla única 2560x1440, Python 3.13.
Resultados:
Comprobación | Método | Resultado |
Captura de pantalla |
| 2560x1440 capturado, reducido a 1400x788 |
Precisión del puntero | 5 puntos objetivo, leídos a través de XWayland | 5/5 exacto, error de 0px |
Botones y desplazamiento | Registro de eventos |
|
Arrastre | Conteo de movimiento | 26 eventos |
Escritura | Tanto | El texto llegó a un editor enfocado |
Acordes |
| Aplicado |
Clic vía MCP | Viaje de ida y vuelta completo por stdio contra una app nativa de Wayland | Se abrió una nueva pestaña |
No probado y vale la pena saberlo: diseños de múltiples monitores, escalado HiDPI, diseños de teclado no estadounidenses en la ruta de códigos de tecla crudos (wtype los maneja correctamente; los códigos de tecla uinput asumen US), y cada compositor que no sea COSMIC.
Se agradecen informes de otras plataformas: desktop_info vuelca todo lo necesario para diagnosticar un problema de backend.
Instalación
git clone <this repo> && cd desktop-mcp
uv venv && uv pip install -e .macOS, Windows y Linux/X11 también necesitan las dependencias de entrada:
uv pip install -e ".[gui]"Linux: configuración de permisos única
Inyectar entrada en Wayland significa escribir en /dev/uinput, que es propiedad de root por defecto:
bash scripts/setup-linux.sh # needs sudo once
# then log out and back inmacOS: configuración de permisos única
Concede Grabación de pantalla y Accesibilidad a la aplicación que inicie el servidor (Terminal, iTerm, Claude Desktop) en Configuración del sistema → Privacidad y seguridad.
Conéctalos
Claude Code:
claude mcp add desktop -- /absolute/path/to/desktop-mcp/.venv/bin/python -m desktop_mcp.serverClaude Desktop (claude_desktop_config.json) y Cursor (.cursor/mcp.json) usan la misma forma:
{
"mcpServers": {
"desktop": {
"command": "/absolute/path/to/desktop-mcp/.venv/bin/python",
"args": ["-m", "desktop_mcp.server"]
}
}
}Herramientas
Herramienta | Propósito |
| Capturar el escritorio |
| Clic izquierdo/derecho/medio, simple o doble |
| Mover el puntero sin hacer clic (hover) |
| Pulsar, mover, soltar: selecciones, deslizadores, movimientos de ventana |
| Desplazamiento de rueda bajo un punto |
| Escribir en el elemento enfocado |
| Acordes como |
| Dónde está el puntero |
| Ventanas abiertas, donde el sistema operativo lo permita |
| Pausar y luego mirar de nuevo |
| Entorno, backends activos, ajustes de seguridad |
Coordenadas
Las capturas de pantalla se reducen (borde largo de 1400px por defecto) antes de enviarse, por lo que el modelo trabaja en píxeles de imagen; el servidor los mapea de vuelta a píxeles de pantalla en cada llamada. Pasa exactamente las coordenadas que lees de la captura.
Este es el detalle que la mayoría de las herramientas caseras de uso de computadora fallan. Si lo omites, cada clic aterriza en una fracción consistente de donde se apuntaba.
Por qué /dev/uinput en lugar de ydotool en Linux
Las compilaciones de ydotool comúnmente empaquetadas exponen un puntero relativo, por lo que un salto a (1280, 720) se entrega como un gran movimiento relativo, que el compositor pasa por la aceleración del puntero. El cursor aterriza cerca del objetivo en lugar de sobre él, y el error crece con la distancia. Un dispositivo uinput absoluto evita la aceleración: medido 5/5 exacto en una pantalla de 2560x1440. ydotool sigue siendo un respaldo cuando /dev/uinput no es escribible.
Tiempo de clic
Los compositores despachan el movimiento del puntero de forma asíncrona, y los toolkits ignoran una pulsación de botón que llega antes de que hayan procesado el movimiento que puso el cursor sobre el widget. Medido en COSMIC: un intervalo de 30ms entre mover y pulsar resalta de manera confiable un botón pero nunca lo activa: el estado de hover es prueba de que las coordenadas son correctas, lo que hace que este fallo sea fácil de malinterpretar como un error de apuntado. 250ms lo activa. Una pulsación también debe mantenerse (~90ms) para registrarse como un clic real en lugar de un evento suelto.
Ambos retrasos son ajustables; bájalos solo si tu escritorio lo tolera.
Seguridad
Esto le da a un modelo el mismo alcance sobre la máquina que la persona sentada frente a ella. Puede leer cualquier cosa en pantalla — administradores de contraseñas abiertos, mensajes privados, tokens — y hacer clic en cualquier cosa, incluidos botones destructivos. Trata una sesión como si le hubieras entregado tu teclado a alguien.
Variables de entorno:
Variable | Por defecto | Efecto |
| off | Capturas de pantalla permitidas, toda entrada rechazada |
|
| Máximo de acciones de entrada por minuto |
| — | Acordes adicionales separados por comas a rechazar |
| — | Añade cada acción a este archivo |
|
| Borde largo de la captura en píxeles |
|
| Segundos entre mover el puntero y pulsar |
|
| Segundos que se mantiene presionado un botón del ratón |
ctrl+alt+F1–F12 siempre están bloqueados; el cambio de VT puede sacar la sesión gráfica de debajo de ti.
Una configuración de revisor de solo lectura:
DESKTOP_MCP_READ_ONLY=1 python -m desktop_mcp.serverVerificar
.venv/bin/python scripts/mcp_smoke.py # real stdio MCP round tripAvailable Tools
11 toolsclickB
Click at (x, y), in screenshot-image coordinates.
button: "left", "right" or "middle". count: 2 for a double-click.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| count | No | ||
| button | No | left | |
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action and coordinate system, but does not mention side effects, return values (e.g., whether a screenshot is returned), or prerequisites like window focus or timeouts. It is not misleading but very minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the core action front-loaded. It includes parameter details in a compact format without any redundant text. Every sentence earns its place, making it appropriately concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of a click tool and no output schema, the description covers the essential action and key parameters. However, it omits potential context such as what happens after the click (does it capture a screenshot?), whether the cursor is moved, and any error conditions. For a tool with no annotations, this is a moderate gap but not severe.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It clarifies the meaning of x and y by specifying they are in screenshot-image coordinates, and it explicitly explains button ('left', 'right', 'middle') and count ('2 for a double-click'). However, the screenshot parameter is not explained, and the count semantics beyond 2 are unspecified. Overall, it adds meaningful value to most parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Click') and a coordinate system ('screenshot-image coordinates'), which clearly identifies the tool's purpose. It distinguishes from sibling tools like move, drag, or scroll by focusing on a simple click action, though it does not explicitly name an alternative. The mention of button and count further specifies the behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like click vs. double-click via count, or when to use drag or scroll instead. It does not mention any exclusions or preferred contexts. The only usage hint is the parameter explanation, but that is about how to invoke, not when.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cursor_positionA
Report where the pointer is, in both image and screen coordinates.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It discloses that the tool reports in both image and screen coordinates, which is useful. However, it doesn't mention that it's read-only (though obvious), nor the output structure, though an output schema exists. It adequately covers the main behavior but could add a note about no side effects or state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, concise sentence that front-loads the core action and covers both coordinate systems. No filler, no redundancy. Reads clearly and efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with a descriptive output schema (not shown), the description covers the essential purpose. It doesn't mention edge cases like whether the cursor must be within a window, but that's likely covered by the output schema. The description is complete for a typical query tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4 per the rubric. No parameter explanation is needed. The description correctly focuses on the output rather than inputs, which is appropriate for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Report') and a clear resource (the pointer position), and specifies two coordinate systems (image and screen). This makes the tool's function immediately clear and distinguishes it from sibling tools like click or move, which are actions rather than queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used to get the current pointer location, but it does not explicitly say when to use it versus alternatives like screenshot or desktop_info. It also doesn't mention that it's a read-only query or that it's useful before other actions. Fine for a simple tool, but lacks guidance on when to choose this over siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
desktop_infoA
Report the environment, the chosen backends, and the active safety limits.
Worth calling once at the start of a session, or whenever something fails.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses exactly what the tool returns (environment, backends, safety limits) and is implicitly a read-only reporting operation. It does not mention side effects or error behavior, but for a simple info tool that's acceptable. It adds value beyond the empty schema by defining the semantic content.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the core purpose, the second provides usage timing. It is front-loaded with the key information and contains zero fluff. Every word earns its place. Ideal length for a tool of this simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter info tool with no output schema, this description is complete. It explains what the tool does and when to call it. The agent has everything needed to decide whether to invoke it and what to expect. No further clarification is required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the schema is empty with 100% schema description coverage by default. Per the rubric, the baseline is 4 when there are no parameters. The description doesn't need to add parameter-specific semantics, and none are present. There is nothing to improve here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Report') and a clear resource: 'the environment, the chosen backends, and the active safety limits.' This is unambiguous and distinguishes it from all sibling tools, which are action-oriented (click, type, etc.). It clearly identifies what information is returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises when to call it: 'Worth calling once at the start of a session, or whenever something fails.' This gives clear timing context. It does not explicitly list alternatives, but given the sibling set is all action tools, it's obvious this is the dedicated info tool. Could be improved with a 'when not to use' statement, but it provides solid guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dragB
Press at one point, move, and release at another.
Use for selecting text, moving windows, and dragging sliders.
| Name | Required | Description | Default |
|---|---|---|---|
| to_x | Yes | ||
| to_y | Yes | ||
| button | No | left | |
| from_x | Yes | ||
| from_y | Yes | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It describes only the physical gesture and does not mention what the tool returns (despite a 'screenshot' parameter defaulting to true), whether the action is reversible, or any side effects on the UI. This is a significant gap for a tool that manipulates the system state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences and front-loaded with the essential action. It is appropriately concise with no filler, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters, no parameter descriptions, and no annotations, this description is insufficient. It omits coordinate conventions, button values, and return behavior, leaving the agent with too many unknowns to call the tool reliably.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description provides no parameter details. It does not clarify the coordinate system (screen vs window-relative), units, the meaning of the 'button' parameter, or the 'screenshot' parameter. An agent has to guess these crucial details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the drag action (press, move, release) and provides concrete use cases (selecting text, moving windows, dragging sliders), which sufficiently distinguishes it from sibling tools like click and move. The verb and resource are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit use cases ('Use for selecting text, moving windows, and dragging sliders') that tell the agent when to invoke it. However, it does not state when not to use it or name alternatives, so some inference is required, but the guidance is still clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
keyB
Press a key or chord, e.g. "enter", "ctrl+s", "alt+tab", "cmd+space".
Modifiers: ctrl, alt, shift, super (aka win), cmd. "cmd" is Command on macOS and Super elsewhere, so one chord works across platforms.
| Name | Required | Description | Default |
|---|---|---|---|
| combo | Yes | ||
| presses | No | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It explains the core key press action and the cross-platform behavior of 'cmd', but omits the 'screenshot' parameter (default true) which implies the tool may capture a screenshot after pressing, and does not mention 'presses' effects. These are significant undocumented behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with two short sentences. The core purpose and examples are front-loaded, and the modifier explanation is directly relevant. No redundant content exists.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool, the description covers the primary action but misses the screenshot side effect and press count, which are important for correct usage. The lack of output schema and annotations means the description should be more thorough to compensate, but it is not.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain all parameters. It elaborates on 'combo' through examples and modifier details, but gives no explanation for 'presses' or 'screenshot'. The description only partially compensates for the schema's lack of information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool presses a key or chord, with concrete examples like 'enter', 'ctrl+s', 'alt+tab'. This verb+resource definition is specific and distinguishes it from sibling tools like 'type_text' (which types text) and 'click' (which clicks).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for keyboard shortcuts and chords, and explains modifiers. However, it does not explicitly contrast with alternatives or state when not to use it (e.g., for typing text). Usage context is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_windowsA
List open windows with their titles and geometry, where the OS allows it.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It mentions 'where the OS allows it', acknowledging platform limitations, but does not detail what happens on unsupported systems (e.g., empty list, error) or clarify that it is a read-only, side-effect-free operation. This leaves some ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the action and output. It contains no unnecessary words and is directly usable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters and no output schema, the description adequately conveys the essential behavior and its limitation. It could explicitly state the return format (list) but 'List' implies that. Given the low complexity, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema coverage (trivial empty schema), the description has no parameter details to add. The baseline of 4 applies because there are no parameters to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (list), resource (open windows), and what it returns (titles and geometry). This clearly distinguishes it from sibling tools that perform actions (click, drag, type_text) or capture screenshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. While the purpose is clear, there is no mention of when listing windows is appropriate, such as before automation to determine target coordinates, or any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
moveB
Move the pointer to (x, y) without clicking -- useful for hover states.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states that the tool moves the pointer without clicking; it omits any mention of the screenshot parameter, whether the move is instantaneous or animated, or any side effects (e.g., triggering hover effects). No information on coordinate system or edge cases is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence with no filler. The core purpose is front-loaded, and the differentiating 'without clicking' appears early. It is appropriately minimal, though the screenshot parameter is not mentioned in the description, which is a minor structural omission.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a pointer-move tool with no annotations and an unexplained screenshot parameter, the description is incomplete. It does not clarify the coordinate origin, whether coordinates are relative to the screen or window, or what the screenshot parameter does. The tool is simple enough that the missing info is notable but not catastrophic, yet the description leaves the agent guessing on a parameter that could affect behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description adds no meaning to any of the parameters. It mentions (x, y) but that is already in the schema as integers. The screenshot parameter is completely unexplained — the description does not hint at when it should be set to true or what it does. The description adds zero value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (move pointer), the target (to x,y), and the key differentiator from siblings: 'without clicking'. It explicitly mentions a use case ('useful for hover states'), which immediately distinguishes it from the click, drag, and scroll tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a usage scenario (hover states) but does not explicitly say when not to use this tool versus alternatives like click or drag. It contrasts with clicking but offers no explicit exclusions or alternative routing beyond the implicit 'without clicking' context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotA
Capture the desktop and return it as an image.
Take one before acting, and after any action whose result you need to see. All other tools expect coordinates measured on this image.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral disclosure. It implies a read-only operation (capturing an image) but does not explicitly state safety, side effects, or limitations such as screen resolution or multi-monitor behavior. The description adds context about the return type but omits potential caveats.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences and every word earns its place. It prioritizes the core action first and then provides timely usage guidance, with no fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with no output schema, the description covers the essential points: what it does, when to use it, and why it matters for other tools. It could mention whether it captures the entire desktop or the primary monitor, but the core usage is well covered given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool accepts no parameters, and the input schema confirms this. Since there are no parameters to document, the description correctly says nothing about them, and the baseline for zero-parameter tools is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Capture the desktop') and the resulting output ('return it as an image'). It clearly distinguishes itself from sibling tools like click, move, and type_text, which all operate on coordinates relative to this image.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs when to take a screenshot: 'before acting' and 'after any action whose result you need to see.' It also tells agents why this matters by noting that 'all other tools expect coordinates measured on this image,' which connects to how the tool should be used in the workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scrollA
Scroll under the pointer at (x, y).
dy is in wheel detents: negative scrolls down the page, positive scrolls up.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| dx | No | ||
| dy | No | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explains dy semantics (negative scrolls down, positive scrolls up), which adds value beyond the schema. However, it does not disclose other behaviors like the default screenshot or dx handling. Given no annotations, the description carries the full burden but is incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core purpose, no wasted words. It efficiently communicates the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple scroll action, the description covers the core (location and vertical direction) but leaves dx and screenshot semantics unspecified. It does not mention return value, but no output schema exists, so that is not strictly required. Still, the screenshot default behavior should be disclosed for full completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explicitly explains dy, and x,y are self-evident from the phrase 'at (x,y)'. However, dx and screenshot parameters are not described, and with 0% schema coverage, the description only partially compensates. The lack of explanation for screenshot (default true) could lead to unexpected behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Scroll'), a target ('under the pointer at (x, y)'), and a clear action. It is distinct from sibling tools like click, move, and drag, even without explicit comparison, because the scroll action is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like drag or key. There are no exclusions, prerequisites, or situational hints. The description only explains parameter semantics, not usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
type_textA
Type text into whatever currently has keyboard focus.
Click the target field first. Newlines in the text are typed as Enter.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the burden of behavioral disclosure. It reveals the newline-as-Enter behavior and the focus requirement, but it does not mention the effect of the 'screenshot' parameter (e.g., whether a screenshot is taken after typing) or what happens if no element is focused. It adds some value but leaves key behavioral aspects undocumented.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core action and then a concise usage/behavior note. Every word earns its place—no fluff, no redundancy, and the critical 'click first' instruction is immediately actionable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 2-parameter tool with no output schema and no annotations, the description covers the main action, the focus requirement, and the newline behavior. However, the screenshot parameter's purpose is left completely unexplained, which is a notable omission given it is optional and defaults to true. The description is adequate for basic use but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It clearly implies that 'text' is the string to type, but it provides zero explanation of the 'screenshot' parameter, which is optional and defaults to true. This is a significant gap because the screenshot behavior is not mentioned at all, leaving the agent uncertain about its purpose or when to disable it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('type') and resource ('text into whatever currently has keyboard focus'), clearly distinguishing it from siblings like click, key, and move. The focus qualifier adds precision, making the tool's scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear prerequisite ('Click the target field first') and defines how newlines are handled, which informs when to use this tool (for text input into focused fields). It does not explicitly name alternatives or exclusions, but the focus-based instruction implies a compare-and-contrast with click or key tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
waitC
Pause, then look again. Use while a page loads or an animation finishes.
| Name | Required | Description | Default |
|---|---|---|---|
| seconds | No | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Pause, then look again,' which hints at a screenshot but doesn't disclose that the pause is blocking, how the screenshot is handled, or that the screenshot is optional (though the schema implies it). No side effects or return behavior are mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler. It places the core action ('Pause') first and follows with a usage hint. It is appropriately brief, though possibly too terse given the missing details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two optional parameters and no output schema, the description still falls short. It doesn't clarify the screenshot behavior (when it's taken, whether it's after the pause), the meaning of default values, or what the tool returns. An agent would need to infer too much from the parameter names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides no explanation of the 'seconds' or 'screenshot' parameters, nor their defaults. The parameter names are self-explanatory, but the description adds zero semantic value beyond the schema's field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb ('Pause') and a resource context (page loads, animations), and the phrase 'look again' implies a screenshot. However, it's vague about whether the operation is purely a delay or also includes a capture, and it doesn't differentiate clearly from the sibling 'screenshot' tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use while a page loads or an animation finishes,' providing a concrete when-to-use scenario. It doesn't mention when not to use it or alternatives, but the context given is specific enough for an agent to recognize the situation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
11 tool updates
v0.1.0- First observed
click - First observed
cursor_position - First observed
desktop_info - First observed
drag - First observed
key - First observed
list_windows - First observed
move - First observed
screenshot - First observed
scroll - First observed
type_text - First observed
wait
TDQS
Scored across 11 tools
Each tool maps to a distinct input modality or query—screenshot, pointer actions, keyboard actions, and state queries—so there is little chance of selecting the wrong tool. The only adjacent pair is type_text vs key, but their descriptions clearly separate literal text entry from chords/hotkeys.
Most names are short, lowercase imperative verbs (click, move, drag, scroll, wait), with a few compound state/query names (cursor_position, desktop_info) and two verb_noun pairs (type_text, list_windows). The style is readable and predictable, though not every tool follows a uniform verb_noun pattern.
11 tools cover the core desktop-automation surface without redundancy or bloat. Each tool addresses a distinct need, so the count feels intentional and well-scoped.
The set provides observation (screenshot, cursor_position, list_windows, desktop_info), mouse control (click, move, drag, scroll), keyboard control (type_text, key), and a synchronization primitive (wait), covering the full cycle of GUI automation. I don't see an obvious missing operation that would leave an agent stuck in normal desktop flows.
Maintenance
Related MCP Connectors
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Web scraping, browser automation, screenshots and AI data extraction for AI agents via MCP.
A cloud Linux desktop for your AI: browse, click, type, read the screen and run commands.
Operate Linux, macOS and Windows from your LLM. Every action runs through an auditable allowlist.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceControls a real Linux desktop from any MCP host. Reads accessibility trees, takes screenshots, and drives clicks, scrolls, and keystrokes across GNOME, KDE/KWin, Hyprland, i3, and COSMIC.2,939 npm801MIT
- AlicenseNot gradedqualityDmaintenanceEnables full Linux desktop control including windows, mouse, keyboard, clipboard, audio, screenshots, OCR, accessibility, and system management through MCP-compatible AI agents.1MIT
- FlicenseNot gradedqualityCmaintenanceAn MCP server that gives a model eyes and hands on a Linux Wayland desktop, enabling screenshot capture, mouse/keyboard control, OCR, and icon detection via OmniParser.1-
- AlicenseNot gradedqualityCmaintenanceMCP server for controlling Linux desktops over Wayland, enabling AI agents to perform mouse, keyboard, window, and screenshot operations on Fedora KDE Plasma.AGPL 3.0