desktop-mcp
desktop-mcp
LLM을 위한 데스크톱 비전 및 제어 도구로, Wayland를 포함한 Linux에서 동작합니다 — 모델이 전체 화면을 스크린샷으로 캡처하고, 보이는 화면을 클릭하고 타이핑하고 스크롤합니다. stdio를 통한 일반 MCP이므로 Claude Code, Claude Desktop, Cursor 또는 다른 MCP 클라이언트에서도 동작합니다.
먼저 읽어보세요: 이 도구가 필요 없을 수도 있습니다
Anthropic은 computer use를 기본으로 제공하며, 이 기능이 동작하는 환경에서는 이 도구 대신 그것을 사용해야 합니다. 통합이 더 잘 되어 있고 실제로 유지관리되기 때문입니다:
macOS 및 Windows: Claude Desktop에는 Cowork와 Claude Code에 computer use가 내장되어 있습니다. 실제 데스크톱을 제어합니다. (문서)
자체 에이전트 구축: Claude API에는 퍼스트파티
computer도구가 있습니다 (computer_toolset_20260801툴셋으로 GA). (문서)
이 도구가 메우는 공백은 Linux입니다. 2026년 8월 현재 computer use는 Linux 데스크톱 앱에서 사용할 수 없으며, Wayland에서의 데스크톱 자동화는 X11에서는 결코 없었던 방식으로 정말로 까다롭습니다:
COSMIC의 컴포지터는
wlr-screencopy를 구현하지 않으므로grim은 완전히 실패하며, 모든 스크린샷 도구는 컴포지터별 경로가 필요합니다.커서 위치를 조회하는 프로토콜도, 다른 클라이언트의 창을 열거하는 프로토콜도 없습니다.
입력은
/dev/uinput을 통해 컴포지터 아래로 들어가야 합니다. 일반적으로 패키징된ydotool은 상대 포인터를 노출하는데, 긴 점프는 포인터 가속에 의해 왜곡되기 때문입니다.
macOS 또는 Windows를 사용 중이라면 이 저장소는 대부분 중복입니다. Linux — 특히 Wayland — 를 사용 중이라면 이것이 실제로 동작하는 선택지입니다.
데스크톱 자동화 MCP 서버는 새로운 범주가 아닙니다. 수천 개의 MCP 서버가 있습니다. 여기서 특별한 점은 Wayland 네이티브 절대-uinput 입력 경로이며, COSMIC에서 픽셀 단위로 정확함이 측정되었다는 것입니다.
Related MCP server: mcp-linux-desktop
상태
크로스 플랫폼으로 작성되었지만 실제로 검증된 구성은 하나뿐입니다. 나머지는 코드 경로일 뿐 증거가 아닙니다 — 그에 맞게 취급하세요.
플랫폼 | 스크린샷 | 입력 | 테스트됨 |
Linux / Wayland |
|
| 예 — COSMIC 전용 |
Linux / X11 |
|
| 아니요 |
macOS |
|
| 아니요 |
Windows |
|
| 아니요 |
테스트된 내용과 방법
다음 환경에서 엔드투엔드로 검증했습니다:
Pop!_OS 24.04 LTS, 커널 7.0.11, COSMIC (
cosmic-comp1.0.0), Wayland, 단일 2560x1440 디스플레이, Python 3.13.
결과:
검사 항목 | 방법 | 결과 |
스크린샷 |
| 2560x1440 캡처, 1400x788로 축소 |
포인터 정확도 | XWayland을 통해 읽은 5개 목표 지점 | 5/5 정확, 0px 오류 |
버튼 및 스크롤 |
| 버튼 1, 3, 4에 대한 |
드래그 |
| 누름과 놓음 사이 26개의 |
타이핑 |
| 텍스트가 포커스된 편집기에 입력됨 |
단축키 조합 | 편집기에서 | 적용됨 |
MCP를 통한 클릭 | 네이티브 Wayland 앱에 대한 전체 stdio 왕복 | 새 탭이 열림 |
테스트되지 않았고 알아둘 만한 사항: 다중 모니터 구성, HiDPI 배율, 원시 키코드 경로에서의 비US 키보드 레이아웃(wtype은 이를 올바르게 처리하지만 uinput 키코드는 US를 가정합니다), 그리고 COSMIC이 아닌 모든 컴포지터.
다른 플랫폼에서의 보고를 환영합니다 — desktop_info는 백엔드 문제를 진단하는 데 필요한 모든 것을 출력합니다.
설치
git clone <this repo> && cd desktop-mcp
uv venv && uv pip install -e .macOS, Windows 및 Linux/X11에는 입력 의존성도 필요합니다:
uv pip install -e ".[gui]"Linux: 1회성 권한 설정
Wayland에서 입력을 주입하려면 /dev/uinput에 쓰기가 필요하며, 기본적으로 이 파일의 소유자는 root입니다:
bash scripts/setup-linux.sh # needs sudo once
# then log out and back inmacOS: 1회성 권한 설정
서버를 실행하는 앱(Terminal, iTerm, Claude Desktop)에 화면 기록 및 손쉬운 사용 권한을 System Settings → Privacy & Security에서 부여하세요.
연결하기
Claude Code:
claude mcp add desktop -- /absolute/path/to/desktop-mcp/.venv/bin/python -m desktop_mcp.serverClaude Desktop (claude_desktop_config.json) 및 Cursor
(.cursor/mcp.json)도 동일한 형식을 사용합니다:
{
"mcpServers": {
"desktop": {
"command": "/absolute/path/to/desktop-mcp/.venv/bin/python",
"args": ["-m", "desktop_mcp.server"]
}
}
}도구
도구 | 용도 |
| 데스크톱 캡처 |
| 왼쪽/오른쪽/가운데 클릭, 단일 또는 더블 |
| 클릭 없이 포인터 이동 (호버) |
| 누르고, 이동하고, 놓기 — 선택, 슬라이더, 창 이동 |
| 특정 지점에서 휠 스크롤 |
| 포커스된 요소에 타이핑 |
|
|
| 포인터의 위치 |
| OS가 허용하는 경우 열린 창 목록 |
| 일시 정지 후 다시 확인 |
| 환경, 활성 백엔드, 안전 설정 |
좌표
스크린샷은 전송 전에 축소되며(기본적으로 긴 변 1400px), 따라서 모델은 이미지 픽셀 단위로 작업합니다. 서버는 모든 호출에서 이를 화면 픽셀로 다시 매핑합니다. 스크린샷에서 읽은 좌표를 그대로 전달하세요.
이것은 대부분의 자체 제작 computer-use 도구가 틀리는 세부 사항입니다. 이를 놓치면 모든 클릭이 목표 지점의 일정한 비율만큼 어긋난 위치에 떨어집니다.
Linux에서 ydotool 대신 /dev/uinput을 사용하는 이유
일반적으로 패키징된 ydotool 빌드는 상대 포인터를 노출하므로 (1280, 720)으로의 점프는 하나의 큰 상대 이동으로 전달되며, 컴포지터는 이를 포인터 가속에 통과시킵니다. 커서는 목표 지점 위가 아니라 근처에 도착하며, 오류는 거리에 따라 커집니다. 절대 uinput 장치는 가속을 우회합니다: 2560x1440 화면에서 5/5 정확함이 측정되었습니다. /dev/uinput에 쓸 수 없을 때 ydotool은 여전히 폴백으로 남아 있습니다.
클릭 타이밍
컴포지터는 포인터 이동을 비동기적으로 전달하며, 툴킷은 커서를 위젯 위에 놓은 이동을 처리하기 전에 도착한 버튼 누름을 무시합니다. COSMIC에서 측정한 결과: 이동과 누름 사이 30ms 간격은 버튼을 강조하기는 하지만 활성화하지는 않습니다 — 호버 상태는 좌표가 정확하다는 증거이므로, 이 실패를 타겟팅 버그로 오인하기 쉽습니다. 250ms면 활성화됩니다. 또한 누름은 오작동 이벤트가 아닌 실제 클릭으로 인식되도록 (~90ms) 동안 유지되어야 합니다.
두 지연 시간 모두 조정 가능합니다. 데스크톱이 허용하는 경우에만 낮추세요.
안전
이 도구는 모델에게 그 자리에 앉아 있는 사람과 동일한 수준의 시스템 접근 권한을 부여합니다. 화면의 모든 것을 읽을 수 있고 — 열려 있는 비밀번호 관리자, 개인 메시지, 토큰 — 파괴적인 버튼을 포함한 무엇이든 클릭할 수 있습니다. 세션을 마치 누군가에게 키보드를 건네준 것처럼 취급하세요.
환경 변수:
변수 | 기본값 | 효과 |
| 꺼짐 | 스크린샷 허용, 모든 입력 거부 |
|
| 분당 최대 입력 동작 수 |
| — | 거부할 추가 단축키 조합(쉼표로 구분) |
| — | 모든 동작을 이 파일에 추가 |
|
| 스크린샷 긴 변(픽셀) |
|
| 포인터 이동과 누름 사이의 시간(초) |
|
| 마우스 버튼을 누르고 있는 시간(초) |
ctrl+alt+F1–F12는 항상 차단됩니다. VT 전환은 사용 중인 그래픽 세션을 갑자기 중단시킬 수 있습니다.
읽기 전용 검토자 설정:
DESKTOP_MCP_READ_ONLY=1 python -m desktop_mcp.server검증
.venv/bin/python scripts/mcp_smoke.py # real stdio MCP round tripAvailable Tools
11 toolsclickB
Click at (x, y), in screenshot-image coordinates.
button: "left", "right" or "middle". count: 2 for a double-click.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| count | No | ||
| button | No | left | |
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action and coordinate system, but does not mention side effects, return values (e.g., whether a screenshot is returned), or prerequisites like window focus or timeouts. It is not misleading but very minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, with the core action front-loaded. It includes parameter details in a compact format without any redundant text. Every sentence earns its place, making it appropriately concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of a click tool and no output schema, the description covers the essential action and key parameters. However, it omits potential context such as what happens after the click (does it capture a screenshot?), whether the cursor is moved, and any error conditions. For a tool with no annotations, this is a moderate gap but not severe.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It clarifies the meaning of x and y by specifying they are in screenshot-image coordinates, and it explicitly explains button ('left', 'right', 'middle') and count ('2 for a double-click'). However, the screenshot parameter is not explained, and the count semantics beyond 2 are unspecified. Overall, it adds meaningful value to most parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Click') and a coordinate system ('screenshot-image coordinates'), which clearly identifies the tool's purpose. It distinguishes from sibling tools like move, drag, or scroll by focusing on a simple click action, though it does not explicitly name an alternative. The mention of button and count further specifies the behavior.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like click vs. double-click via count, or when to use drag or scroll instead. It does not mention any exclusions or preferred contexts. The only usage hint is the parameter explanation, but that is about how to invoke, not when.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
cursor_positionA
Report where the pointer is, in both image and screen coordinates.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full behavioral burden. It discloses that the tool reports in both image and screen coordinates, which is useful. However, it doesn't mention that it's read-only (though obvious), nor the output structure, though an output schema exists. It adequately covers the main behavior but could add a note about no side effects or state changes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, concise sentence that front-loads the core action and covers both coordinate systems. No filler, no redundancy. Reads clearly and efficiently.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with a descriptive output schema (not shown), the description covers the essential purpose. It doesn't mention edge cases like whether the cursor must be within a window, but that's likely covered by the output schema. The description is complete for a typical query tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so the baseline is 4 per the rubric. No parameter explanation is needed. The description correctly focuses on the output rather than inputs, which is appropriate for a parameterless tool.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Report') and a clear resource (the pointer position), and specifies two coordinate systems (image and screen). This makes the tool's function immediately clear and distinguishes it from sibling tools like click or move, which are actions rather than queries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used to get the current pointer location, but it does not explicitly say when to use it versus alternatives like screenshot or desktop_info. It also doesn't mention that it's a read-only query or that it's useful before other actions. Fine for a simple tool, but lacks guidance on when to choose this over siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
desktop_infoA
Report the environment, the chosen backends, and the active safety limits.
Worth calling once at the start of a session, or whenever something fails.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses exactly what the tool returns (environment, backends, safety limits) and is implicitly a read-only reporting operation. It does not mention side effects or error behavior, but for a simple info tool that's acceptable. It adds value beyond the empty schema by defining the semantic content.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the core purpose, the second provides usage timing. It is front-loaded with the key information and contains zero fluff. Every word earns its place. Ideal length for a tool of this simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter info tool with no output schema, this description is complete. It explains what the tool does and when to call it. The agent has everything needed to decide whether to invoke it and what to expect. No further clarification is required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, and the schema is empty with 100% schema description coverage by default. Per the rubric, the baseline is 4 when there are no parameters. The description doesn't need to add parameter-specific semantics, and none are present. There is nothing to improve here.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Report') and a clear resource: 'the environment, the chosen backends, and the active safety limits.' This is unambiguous and distinguishes it from all sibling tools, which are action-oriented (click, type, etc.). It clearly identifies what information is returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises when to call it: 'Worth calling once at the start of a session, or whenever something fails.' This gives clear timing context. It does not explicitly list alternatives, but given the sibling set is all action tools, it's obvious this is the dedicated info tool. Could be improved with a 'when not to use' statement, but it provides solid guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dragB
Press at one point, move, and release at another.
Use for selecting text, moving windows, and dragging sliders.
| Name | Required | Description | Default |
|---|---|---|---|
| to_x | Yes | ||
| to_y | Yes | ||
| button | No | left | |
| from_x | Yes | ||
| from_y | Yes | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It describes only the physical gesture and does not mention what the tool returns (despite a 'screenshot' parameter defaulting to true), whether the action is reversible, or any side effects on the UI. This is a significant gap for a tool that manipulates the system state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences and front-loaded with the essential action. It is appropriately concise with no filler, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 6 parameters, no parameter descriptions, and no annotations, this description is insufficient. It omits coordinate conventions, button values, and return behavior, leaving the agent with too many unknowns to call the tool reliably.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0% and the description provides no parameter details. It does not clarify the coordinate system (screen vs window-relative), units, the meaning of the 'button' parameter, or the 'screenshot' parameter. An agent has to guess these crucial details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the drag action (press, move, release) and provides concrete use cases (selecting text, moving windows, dragging sliders), which sufficiently distinguishes it from sibling tools like click and move. The verb and resource are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit use cases ('Use for selecting text, moving windows, and dragging sliders') that tell the agent when to invoke it. However, it does not state when not to use it or name alternatives, so some inference is required, but the guidance is still clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
keyB
Press a key or chord, e.g. "enter", "ctrl+s", "alt+tab", "cmd+space".
Modifiers: ctrl, alt, shift, super (aka win), cmd. "cmd" is Command on macOS and Super elsewhere, so one chord works across platforms.
| Name | Required | Description | Default |
|---|---|---|---|
| combo | Yes | ||
| presses | No | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully disclose behavior. It explains the core key press action and the cross-platform behavior of 'cmd', but omits the 'screenshot' parameter (default true) which implies the tool may capture a screenshot after pressing, and does not mention 'presses' effects. These are significant undocumented behaviors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with two short sentences. The core purpose and examples are front-loaded, and the modifier explanation is directly relevant. No redundant content exists.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool, the description covers the primary action but misses the screenshot side effect and press count, which are important for correct usage. The lack of output schema and annotations means the description should be more thorough to compensate, but it is not.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain all parameters. It elaborates on 'combo' through examples and modifier details, but gives no explanation for 'presses' or 'screenshot'. The description only partially compensates for the schema's lack of information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool presses a key or chord, with concrete examples like 'enter', 'ctrl+s', 'alt+tab'. This verb+resource definition is specific and distinguishes it from sibling tools like 'type_text' (which types text) and 'click' (which clicks).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for keyboard shortcuts and chords, and explains modifiers. However, it does not explicitly contrast with alternatives or state when not to use it (e.g., for typing text). Usage context is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_windowsA
List open windows with their titles and geometry, where the OS allows it.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It mentions 'where the OS allows it', acknowledging platform limitations, but does not detail what happens on unsupported systems (e.g., empty list, error) or clarify that it is a read-only, side-effect-free operation. This leaves some ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the action and output. It contains no unnecessary words and is directly usable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters and no output schema, the description adequately conveys the essential behavior and its limitation. It could explicitly state the return format (list) but 'List' implies that. Given the low complexity, this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With zero parameters and 100% schema coverage (trivial empty schema), the description has no parameter details to add. The baseline of 4 applies because there are no parameters to explain.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (list), resource (open windows), and what it returns (titles and geometry). This clearly distinguishes it from sibling tools that perform actions (click, drag, type_text) or capture screenshots.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. While the purpose is clear, there is no mention of when listing windows is appropriate, such as before automation to determine target coordinates, or any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
moveB
Move the pointer to (x, y) without clicking -- useful for hover states.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states that the tool moves the pointer without clicking; it omits any mention of the screenshot parameter, whether the move is instantaneous or animated, or any side effects (e.g., triggering hover effects). No information on coordinate system or edge cases is given.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence with no filler. The core purpose is front-loaded, and the differentiating 'without clicking' appears early. It is appropriately minimal, though the screenshot parameter is not mentioned in the description, which is a minor structural omission.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a pointer-move tool with no annotations and an unexplained screenshot parameter, the description is incomplete. It does not clarify the coordinate origin, whether coordinates are relative to the screen or window, or what the screenshot parameter does. The tool is simple enough that the missing info is notable but not catastrophic, yet the description leaves the agent guessing on a parameter that could affect behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the description adds no meaning to any of the parameters. It mentions (x, y) but that is already in the schema as integers. The screenshot parameter is completely unexplained — the description does not hint at when it should be set to true or what it does. The description adds zero value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (move pointer), the target (to x,y), and the key differentiator from siblings: 'without clicking'. It explicitly mentions a use case ('useful for hover states'), which immediately distinguishes it from the click, drag, and scroll tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a usage scenario (hover states) but does not explicitly say when not to use this tool versus alternatives like click or drag. It contrasts with clicking but offers no explicit exclusions or alternative routing beyond the implicit 'without clicking' context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
screenshotA
Capture the desktop and return it as an image.
Take one before acting, and after any action whose result you need to see. All other tools expect coordinates measured on this image.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden of behavioral disclosure. It implies a read-only operation (capturing an image) but does not explicitly state safety, side effects, or limitations such as screen resolution or multi-monitor behavior. The description adds context about the return type but omits potential caveats.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only two sentences and every word earns its place. It prioritizes the core action first and then provides timely usage guidance, with no fluff or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with no output schema, the description covers the essential points: what it does, when to use it, and why it matters for other tools. It could mention whether it captures the entire desktop or the primary monitor, but the core usage is well covered given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool accepts no parameters, and the input schema confirms this. Since there are no parameters to document, the description correctly says nothing about them, and the baseline for zero-parameter tools is high.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Capture the desktop') and the resulting output ('return it as an image'). It clearly distinguishes itself from sibling tools like click, move, and type_text, which all operate on coordinates relative to this image.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly instructs when to take a screenshot: 'before acting' and 'after any action whose result you need to see.' It also tells agents why this matters by noting that 'all other tools expect coordinates measured on this image,' which connects to how the tool should be used in the workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
scrollA
Scroll under the pointer at (x, y).
dy is in wheel detents: negative scrolls down the page, positive scrolls up.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| dx | No | ||
| dy | No | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description explains dy semantics (negative scrolls down, positive scrolls up), which adds value beyond the schema. However, it does not disclose other behaviors like the default screenshot or dx handling. Given no annotations, the description carries the full burden but is incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core purpose, no wasted words. It efficiently communicates the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple scroll action, the description covers the core (location and vertical direction) but leaves dx and screenshot semantics unspecified. It does not mention return value, but no output schema exists, so that is not strictly required. Still, the screenshot default behavior should be disclosed for full completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explicitly explains dy, and x,y are self-evident from the phrase 'at (x,y)'. However, dx and screenshot parameters are not described, and with 0% schema coverage, the description only partially compensates. The lack of explanation for screenshot (default true) could lead to unexpected behavior.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Scroll'), a target ('under the pointer at (x, y)'), and a clear action. It is distinct from sibling tools like click, move, and drag, even without explicit comparison, because the scroll action is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like drag or key. There are no exclusions, prerequisites, or situational hints. The description only explains parameter semantics, not usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
type_textA
Type text into whatever currently has keyboard focus.
Click the target field first. Newlines in the text are typed as Enter.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the burden of behavioral disclosure. It reveals the newline-as-Enter behavior and the focus requirement, but it does not mention the effect of the 'screenshot' parameter (e.g., whether a screenshot is taken after typing) or what happens if no element is focused. It adds some value but leaves key behavioral aspects undocumented.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the core action and then a concise usage/behavior note. Every word earns its place—no fluff, no redundancy, and the critical 'click first' instruction is immediately actionable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 2-parameter tool with no output schema and no annotations, the description covers the main action, the focus requirement, and the newline behavior. However, the screenshot parameter's purpose is left completely unexplained, which is a notable omission given it is optional and defaults to true. The description is adequate for basic use but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It clearly implies that 'text' is the string to type, but it provides zero explanation of the 'screenshot' parameter, which is optional and defaults to true. This is a significant gap because the screenshot behavior is not mentioned at all, leaving the agent uncertain about its purpose or when to disable it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('type') and resource ('text into whatever currently has keyboard focus'), clearly distinguishing it from siblings like click, key, and move. The focus qualifier adds precision, making the tool's scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives a clear prerequisite ('Click the target field first') and defines how newlines are handled, which informs when to use this tool (for text input into focused fields). It does not explicitly name alternatives or exclusions, but the focus-based instruction implies a compare-and-contrast with click or key tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
waitC
Pause, then look again. Use while a page loads or an animation finishes.
| Name | Required | Description | Default |
|---|---|---|---|
| seconds | No | ||
| screenshot | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Pause, then look again,' which hints at a screenshot but doesn't disclose that the pause is blocking, how the screenshot is handled, or that the screenshot is optional (though the schema implies it). No side effects or return behavior are mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler. It places the core action ('Pause') first and follows with a usage hint. It is appropriately brief, though possibly too terse given the missing details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with two optional parameters and no output schema, the description still falls short. It doesn't clarify the screenshot behavior (when it's taken, whether it's after the pause), the meaning of default values, or what the tool returns. An agent would need to infer too much from the parameter names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides no explanation of the 'seconds' or 'screenshot' parameters, nor their defaults. The parameter names are self-explanatory, but the description adds zero semantic value beyond the schema's field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb ('Pause') and a resource context (page loads, animations), and the phrase 'look again' implies a screenshot. However, it's vague about whether the operation is purely a delay or also includes a capture, and it doesn't differentiate clearly from the sibling 'screenshot' tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use while a page loads or an animation finishes,' providing a concrete when-to-use scenario. It doesn't mention when not to use it or alternatives, but the context given is specific enough for an agent to recognize the situation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
11 tool updates
v0.1.0- First observed
click - First observed
cursor_position - First observed
desktop_info - First observed
drag - First observed
key - First observed
list_windows - First observed
move - First observed
screenshot - First observed
scroll - First observed
type_text - First observed
wait
TDQS
Scored across 11 tools
Each tool maps to a distinct input modality or query—screenshot, pointer actions, keyboard actions, and state queries—so there is little chance of selecting the wrong tool. The only adjacent pair is type_text vs key, but their descriptions clearly separate literal text entry from chords/hotkeys.
Most names are short, lowercase imperative verbs (click, move, drag, scroll, wait), with a few compound state/query names (cursor_position, desktop_info) and two verb_noun pairs (type_text, list_windows). The style is readable and predictable, though not every tool follows a uniform verb_noun pattern.
11 tools cover the core desktop-automation surface without redundancy or bloat. Each tool addresses a distinct need, so the count feels intentional and well-scoped.
The set provides observation (screenshot, cursor_position, list_windows, desktop_info), mouse control (click, move, drag, scroll), keyboard control (type_text, key), and a synchronization primitive (wait), covering the full cycle of GUI automation. I don't see an obvious missing operation that would leave an agent stuck in normal desktop flows.
Maintenance
Related MCP Connectors
OCR, transcription, file extraction, and image generation for AI agents via MCP.
Connect ChatGPT and Claude to your Mac, Windows or Linux with secure MCP remote access.
Web scraping, browser automation, screenshots and AI data extraction for AI agents via MCP.
A cloud Linux desktop for your AI: browse, click, type, read the screen and run commands.
Related MCP Servers
- AlicenseNot gradedqualityAmaintenanceControls a real Linux desktop from any MCP host. Reads accessibility trees, takes screenshots, and drives clicks, scrolls, and keystrokes across GNOME, KDE/KWin, Hyprland, i3, and COSMIC.4,587 npm819MIT
- AlicenseNot gradedqualityDmaintenanceEnables full Linux desktop control including windows, mouse, keyboard, clipboard, audio, screenshots, OCR, accessibility, and system management through MCP-compatible AI agents.1MIT
- FlicenseNot gradedqualityCmaintenanceAn MCP server that gives a model eyes and hands on a Linux Wayland desktop, enabling screenshot capture, mouse/keyboard control, OCR, and icon detection via OmniParser.1-
- AlicenseNot gradedqualityCmaintenanceMCP server for controlling Linux desktops over Wayland, enabling AI agents to perform mouse, keyboard, window, and screenshot operations on Fedora KDE Plasma.AGPL 3.0