qt-commander
This MCP server enables AI agents to introspect and automate native Qt applications (both QWidget and QML/Qt Quick) without requiring source code changes — like Playwright for desktop Qt apps.
Process & Session Management: Discover running Qt processes (qt_list_processes), attach to a process to inject the helper library (qt_attach), manage active sessions (qt_list_sessions), and detach (qt_detach).
Build & Environment Setup: Automatically detect MSVC, MinGW, and Qt installations (qt_detect_msvc_and_qt), then build the injection library on demand with your chosen toolchain and Qt version (qt_build).
UI Introspection: Capture the full widget/QML tree as a snapshot with geometry, visibility, z-order, and properties at configurable detail levels (qt_snapshot), occlusion-prune to see what's actually visible (qt_prune_snapshot), find elements by class, text, object name, properties, and depth (qt_find_element), read/write QObject properties (qt_get_property, qt_set_property), invoke methods (qt_call_method), and take screenshots of elements or windows (qt_screenshot).
Mouse Interaction: Simulate real mouse events: click directly on elements (qt_mouse_click), double-click (qt_mouse_dbl_click), click at exact window coordinates with hit-testing (qt_mouse_click_at), click element region centers (qt_mouse_click_region), press/release for drags (qt_mouse_press, qt_mouse_release), move the pointer (qt_mouse_move), scroll (qt_mouse_wheel), and open context menus (qt_mouse_context_menu).
Keyboard Interaction: Type text into elements with optional modifiers (qt_keyboard_input), send key combos like Ctrl+C (qt_key_combo), and programmatically focus elements (qt_focus).
Everything uses real input through Qt's native event pipeline, so applications behave exactly as if a human were interacting.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@qt-commanderAttach to the running Qt app and show its UI snapshot"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
qt-commander
MCP server for Qt application introspection and automation — the Playwright for native Qt, covering both QWidget and QML interfaces.
Why qt-commander
AI agents (Claude, Cursor, …) can drive native Qt applications the way Playwright drives web pages:
No source changes — the library is injected into a running process; you control any Qt app (yours or a third party's) as-is.
Both UI stacks — QWidget and QML/Qt Quick, Qt 5.15 and Qt 6.8, MSVC and MinGW.
What the agent gets — full UI snapshots with geometry, z-order, visibility, opacity and properties; occlusion-pruned views of what a human actually sees; element lookup by text / type / property; real input pipeline clicks, typing, keyboard shortcuts, drags.
Easy to try —
uv run --project <path/to/qt-commander> qt-commander-mcp; the injector and library compile on demand against any detected Qt kit.
Related MCP server: cua-plugin
Quick Start
# Launch MCP server via uv (no global Python install needed — uv resolves
# pyproject.toml / uv.lock and manages an isolated environment; the first
# run auto-syncs the project venv and installs the qt-commander-mcp script)
uv run --project /absolute/path/to/qt-commander qt-commander-mcpRequires uv and Python 3.10+.
MCP client configuration
Claude Code — add to .mcp.json / user-scope MCP config:
{
"mcpServers": {
"qt-commander": {
"command": "uv",
"args": ["run", "--project", "/absolute/path/to/qt-commander", "qt-commander-mcp"]
}
}
}The --project flag pins the project location, so no cwd is needed —
the server starts from any working directory. (A cwd-dependent launch is a
common cause of "MCP connection failed": if the client does not honor cwd,
uv run falls back to the ambient Python and the module is not found.)
Cursor / other MCP clients — same command shape; the server speaks stdio MCP and needs no other setup. See llms-install.md for the full install guide (including removing legacy pip installs).
MCP Tools
Tool | Description |
| List running Qt processes (cross-platform via psutil) |
| Inject library into a target process and open a session |
| Disconnect from a session, optionally eject the library |
| List active sessions |
| Auto-detect MSVC, MinGW toolchains, and Qt installations available for building |
| Compile injector + library on demand — |
| Capture the UI element tree — |
| Occlusion-prune a snapshot: remove elements fully covered by higher-z opaque elements (equal z ordered by creation, later covers earlier; children paint above their parent), mark partially covered ones with |
| Find elements by type, text, or property query |
| Read a QObject property |
| Write a QObject property |
| Invoke a QObject method |
| Capture a screenshot of a specific element or window |
| Send a mouse click to a UI element (direct delivery) |
| Click at an exact window coordinate — routed through the real Qt input pipeline (QPA), with real scene-graph/widget hit testing, identical to a human click |
| Click at the center of an element's on-screen region — real hit testing decides the actual target (e.g. a QML Rectangle's MouseArea) |
| Press a mouse button on an element without releasing it |
| Release a previously pressed mouse button (completes a click or a drag) |
| Move the pointer to an element-local position (drag = press → move → release) |
| Send keyboard input (typed text, optionally with held modifiers) |
| Send a shortcut such as |
| Set focus on a specific element |
Architecture
┌──────────┐ stdio ┌──────────────┐ subprocess ┌──────────────┐
│ AI Agent │ ◄────────────► │ MCP Server │ ──────────────► │ qt-injector │
└──────────┘ │ (Python) │ │ (C++) │
└──────────────┘ └──────┬───────┘
│
CreateRemoteThread
│
┌────────▼───────┐
│ libqt-commander│
│ (C++/Qt) │
└────────────────┘Component | Path | Language | Role |
MCP Server |
| Python | Protocol bridge, session management, on-demand build |
Injector CLI |
| C++ | Standalone binary that loads the library into a target process |
Injection Library |
| C++/Qt | In-process engine for UI introspection, manipulation, capture |
Shared |
| C++ | Frame protocol, TCP socket utilities |
How it works
AI Agent sends an MCP tool call (e.g.
qt_snapshot) via stdio.MCP Server spawns
qt-injector.exeas a subprocess with the target PID.qt-injector loads
libqt-commander.dllinto the target Qt process viaCreateRemoteThread+LoadLibraryW. Before injecting the library it preloads the library's transitive dependency closure (Qt DLLs the target app does not link, e.g. Qt5Widgets for a pure QML app) from the library's own directory — no manual Qt DLL copies next to the target executable are needed. It then performs a token-authenticated handshake and prints the library's TCP port to stdout.MCP Server connects to the library over TCP and relays RPC calls (snapshot, click, input, etc.) using a 4-byte length-prefix frame protocol.
Testing
Everything (C++ unit + E2E suites, pytest, and the deployment-level preload verification) runs from one CMake build tree:
# Single build tree (injector + library + test apps + all tests).
# Both Qt5 and Qt6 are supported; pick the Qt you want to validate:
# Qt5 MSVC: -DQT_MAJOR_VERSION=5 -DQt5_DIR=C:/Qt/5.15.2/msvc2019_64/lib/cmake/Qt5
# Qt6 MSVC: -DQT_MAJOR_VERSION=6 -DQt6_DIR=C:/Qt/6.8.3/msvc2022_64/lib/cmake/Qt6
# Qt6 MinGW: same, but Qt6_DIR=C:/Qt/6.8.3/mingw_64/lib/cmake/Qt6
# with the MinGW toolchain on PATH (see "MinGW" below)
cmake -S . -B build/msvc -G Ninja ^
-DBUILD_INJECTOR=ON -DBUILD_TESTS=ON -DWITH_QML=ON ^
-DCMAKE_BUILD_TYPE=Release -DQT_MAJOR_VERSION=6 ^
-DQt6_DIR=C:/Qt/6.8.3/msvc2022_64/lib/cmake/Qt6
# Build everything, then run ALL tests in one command:
cmake --build build/msvc
ctest --test-dir build/msvc --output-on-failureverify_preload (E2E deployment checks) auto-detects the Qt major AND
toolchain kit (msvc/mingw) of the deployed libqt-commander.dll and
verifies the matching DLL set — even windeployqt follows the deployment's
kit — so the same script validates Qt5/Qt6 × msvc/mingw deployments from
either build tree.
MinGW
MinGW builds are fully supported (Qt5 and Qt6 MinGW kits). Use
qt_build with toolchain="mingw": pass the MinGW toolchain's bin dir
as vcvars_path and the kit's qtenv2.bat as qt_env (MinGW Qt kits
ship qtenv2.bat just like MSVC kits). qt_detect_msvc_and_qt reports
MinGW toolchains (mingw_toolchains) and tags each Qt kit with its
kit ("msvc"/"mingw").
Notes:
Compiler version: Qt 5's official MinGW kit ships GCC 8.1, whose libstdc++ cannot compile
std::filesystemheaders (fixed in 8.3); use a GCC ≥ 9 toolchain (e.g. Qt's bundledmingw1310_64) for Qt 5 too.qt_buildpins the compiler explicitly (-DCMAKE_C/CXX_COMPILER), so other gcc builds on PATH (e.g. a Strawberry Perl toolchain) never get picked up.Runtime DLLs: MinGW executables need
libgcc_s_seh-1.dll,libstdc++-6.dll,libwinpthread-1.dllnext to them. The build deploys the compiler's own runtime (a Qt kit's older runtime lacks newer symbols), andverify_preloadmatches the deployed app's runtime to the library's.Kit matching: the injector library, its deployment, and the target application must share the same Qt kit (all MSVC or all MinGW) — mixing kits loads two Qt module sets into one process and breaks the preload closure.
ctest runs 19 suites: 14 injector C++ suites (including three real E2E
injection suites against the widget test app), 3 library C++ suites, the
full pytest suite (python_unit_tests), and the E2E preload verification
(verify_preload, labeled e2e, which needs the qt_build artifacts in
.qt-commander/bin).
The E2E suites auto-anchor their working directory to their own build
tree (test_util.h::chdir_to_exe_dir), so they produce identical results
when launched directly from the repo root or through ctest.
Quick subsets:
pytest tests/ -q # Python only
ctest --test-dir build/msvc -LE e2e # skip slow E2E
ctest --test-dir build/msvc -R "test_selector" # one suiteTest matrix (verified state, 2026-08)
Suite | Location | Language | Tests | Requires |
Server unit |
| Python | 254 | Python 3.10+ |
Injector unit |
| C++ | 326 | MSVC |
Library unit |
| C++ | 43 | MSVC + Qt |
E2E injection |
| C++ | 48 | MSVC + Qt + test app |
E2E preload |
| Python | 3 scenarios | qt_build artifacts |
Project Structure
qt-commander/
├── qt_commander/ Python MCP server
│ ├── server.py FastMCP app, 22 tools + 2 resources
│ ├── session.py Session/SessionManager with RPC lock
│ ├── rpc_client.py Subprocess injector launcher
│ ├── builder.py On-demand MSVC build orchestrator
│ ├── process_detector.py Cross-platform Qt process discovery
│ ├── environment_detector.py MSVC/Qt build environment auto-detection
│ ├── framing.py 4-byte BE length-prefix frame protocol
│ ├── occlusion.py Snapshot occlusion solving (drop covered
│ │ elements, mark visible ratio)
│ └── errors.py MCP error code registry
│
├── src/
│ ├── common/ Shared C++ utilities
│ │ ├── framing.h Frame protocol (header-only)
│ │ ├── socket_utils.h TCP abstraction
│ │ └── socket_utils.cpp
│ ├── injector/ Standalone injection CLI
│ │ ├── main.cpp Entry point, argument parsing, --list-deps, exit codes 1-6
│ │ ├── injector.h Public API declarations
│ │ ├── injector_win.cpp Win32 implementation (CreateRemoteThread, PE
│ │ │ import parser, dependency-closure preload)
│ │ ├── injector_di.cpp DI variants (IProcessOps-driven, fully testable)
│ │ └── os_ops.h IProcessOps / MockProcessOps / Win32ProcessOps
│ └── library/ Injected DLL
│ ├── entry_win.cpp DllMain / Windows entry
│ ├── api.h InitParams handshake layout (1024 bytes)
│ ├── compat_qt.h Qt5/Qt6 compatibility macros
│ ├── core/ UI scanner, event injector, screenshot, element map
│ ├── rpc/ TCP RPC server (JSON-RPC handler)
│ └── selector/ Element query engine
│
├── tests/
│ ├── unit_server/ Python unit tests (254)
│ ├── unit_injector/ C++ unit + E2E tests (326)
│ ├── unit_library/ C++ library component tests (43)
│ ├── verify_preload.py E2E: dependency preload scenarios A/B/C
│ └── test-apps/ Minimal Qt test applications
│
└── CMakeLists.txtLicense
MIT — free to use, modify, distribute, and integrate into commercial projects, with attribution.
Author
Developed and maintained by TieFeiyu.
Available Tools
25 toolsqt_attachA
Inject the helper library into a running Qt process and open a session.
Find the target pid with qt_list_processes first (only Qt processes are listed, each with its qt_version/arch/bitness). The build artifacts in .qt-commander/bin must match the target process: same Qt major (qt_major used in qt_build), same toolchain (msvc/mingw), same Debug/Release build type, same 64/32-bit — a mismatch makes injection fail with code 2002. A process that is already attached fails with 2006; detach first.
If the build has not been completed, this returns error code 2001 with the steps to follow (qt_detect_msvc_and_qt → AskUserQuestion → qt_build → qt_attach again); it does not build automatically.
Injection modifies the target process (loads a DLL, starts a local RPC thread/listener). If the target process exits, the session goes dead: operations then fail with connection/timeout errors — check qt_list_sessions and re-attach.
| Name | Required | Description | Default |
|---|---|---|---|
| pid | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses side effects: injection modifies the target process (loads DLL, starts RPC thread/listener), session dies with the process, and specific error codes (2001, 2002, 2006) are explained. This is thorough and beyond the basic operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but every sentence carries meaningful information about prerequisites, compatibility, error handling, and side effects. The main action is front-loaded and the structure is logical, though slightly dense.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers all necessary context: how to prepare, compatibility requirements, error codes, side effects, and session lifecycle. Since an output schema exists, return format details are appropriately omitted. It is complete for a tool with these complexities.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema only lists 'pid' as an integer with no description. The description compensates by telling the user to find the pid via qt_list_processes and refers to the target process. This gives enough context for a single simple parameter, though it stops short of a formal definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Inject the helper library into a running Qt process and open a session.' This is specific and distinguishes it from sibling tools like qt_detach and qt_list_processes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit guidance: use qt_list_processes to find the pid, detach before re-attaching if already attached, and check qt_list_sessions after session death. It also clarifies it does not build automatically and gives the tool sequence when build is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_buildA
Build the Qt injection library and injector for the specified environment.
toolchain selects the compiler: "msvc" (default; vcvars_path
is a vcvars bat, qt_env a qtenv2.bat) or "mingw"
(vcvars_path is the MinGW bin dir; qt_env the kit's qtenv2.bat
— MinGW Qt kits ship one too; the Qt-bundled Ninja is used
automatically).
Before calling this tool, the AI MUST present available build
environments (from qt_detect_msvc_and_qt) to the user and let them
choose. Do NOT guess or auto-pick paths — the user decides which VS
or MinGW toolchain and which Qt installation to use.
The AI MUST also ask the user whether to enable QML/QQuick support
(with_qml). QML support is required for inspecting QML/Qt Quick
applications but adds build dependencies.
The AI MUST also ask the user to choose the build_type (Debug or
Release). The build type must match the target process — a Debug
process requires a Debug build, a Release process requires Release.
qt_major (default 5) must match the target application's Qt major
version — read it from qt_list_processes' qt_version field or from
the qt_detect_msvc_and_qt result; a Qt6 target built with qt_major=5
fails to inject. vcvars_args — extra args for the VS vcvars batch
(e.g. "-arch x64"). generator — CMake generator override; omit to
auto-select. Before rebuilding, detach any session that still holds
the old DLL (qt_detach with purge=True) or the build fails on a locked
file.
| Name | Required | Description | Default |
|---|---|---|---|
| qt_env | No | ||
| qt_major | No | ||
| with_qml | No | ||
| generator | No | ||
| toolchain | No | msvc | |
| build_type | No | Release | |
| vcvars_args | No | ||
| vcvars_path | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full behavioral transparency. It explains toolchain selection behavior, automatic Qt-bundled Ninja usage, the consequence of qt_major mismatch (fails to inject), and the locked-file failure mode when old DLLs remain attached. This goes well beyond a simple 'build' statement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Although lengthy, the description is densely packed with necessary operational detail and is well-structured: it starts with a purpose sentence, then explains toolchain semantics, then mandatory user-consent steps, then parameter-specific guidance, and finally a rebuild prerequisite. Every sentence earns its place, and formatting aids scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 8-parameter complexity, absence of annotations, and 0% schema coverage, this description is remarkably complete. It covers all parameters, critical preconditions, user interaction requirements, cross-tool dependencies, and failure modes. The presence of an output schema further reduces the need to document return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description compensates fully by explaining every parameter: toolchain with msvc/mingw semantics, vcvars_path/qt_env roles, qt_major matching rule, with_qml purpose, build_type matching requirement, vcvars_args example, and generator override guidance. It adds meaning well beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Build the Qt injection library and injector for the specified environment.' It clearly distinguishes this build tool from the sibling inspection/control tools like qt_list_processes, qt_attach, and qt_detach. The unique build purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage rules: the AI MUST present build environments to the user, MUST ask about QML support, MUST ask for build_type, and MUST NOT guess paths. It also gives a concrete alternative/prerequisite by directing the AI to use qt_detach with purge=True before rebuilding and to read qt_major from qt_list_processes or qt_detect_msvc_and_qt.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_call_methodA
Invoke a QMetaObject-invokable method on a UI element.
DESTRUCTIVE: directly changes the target application's state with no
undo. Methods like close(), deleteLater() (destroys the
element and invalidates its id) or any app slot execute for real.
Prefer simulating user input (clicks/keys); call methods only when a
direct API invocation is intended. Argument types are coerced
silently (up to 10 args).
| Name | Required | Description | Default |
|---|---|---|---|
| args | No | ||
| method | Yes | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries the full burden. It discloses critical behavioral traits: destructive with no undo, methods execute for real, deleteLater destroys the element and invalidates its id, and args are silently coerced up to 10. This goes beyond the schema and gives essential safety warnings.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, followed by crucial warnings and usage guidance. It is concise with no wasted words; every sentence provides necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return values are covered. The description covers purpose, usage, destructive behavior, and arg constraints. However, it leaves session_id/element_id semantics and failure behavior unexplained, which is a noticeable gap for a 4-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions are 0%, so the description must compensate. It adds meaning for method (invokable, dangerous examples) and args (coerced, max 10), but session_id and element_id remain completely undefined. The description does not fully explain the parameters needed for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool invokes a QMetaObject-invokable method on a UI element, with specific examples (close, deleteLater) and contrasts this with simulating user input. This distinguishes it from sibling input simulation and property tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises preferring simulating user input (clicks/keys) and using direct method calls only when a direct API invocation is intended. This provides clear when-not-to-use guidance and names an alternative category (input simulation).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_detachA
Disconnect from a Qt process session.
purge=True (default False) also ejects the injected DLL from the
target process and deletes the session's saved files (snapshots,
screenshots, session metadata) — irreversible. Use purge=True before
rebuilding the library: otherwise the DLL file stays locked by the
target process and qt_build fails. With purge=False the DLL stays
loaded in the target until that process exits.
| Name | Required | Description | Default |
|---|---|---|---|
| purge | No | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden of behavioral disclosure. It clearly states that purge=True 'ejects the injected DLL' and 'deletes the session's saved files' and marks the action as 'irreversible.' It also discloses the consequence of purge=False: the DLL stays loaded until the process exits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, with the core purpose in the first line. Each subsequent sentence explains the purge behavior and links it to a real-world consequence, with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the purpose, parameter behavior, irreversible side effects, and a practical use case. It doesn't detail return values or error handling, but an output schema exists, and the tool is simple enough that this is not a major gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides only names and types, but the description adds deep semantic meaning for purge: default False, what it does, and why it matters. The session_id parameter is not explicitly described but is implied by the term 'session'; the description's detail on purge compensates for the 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific action: 'Disconnect from a Qt process session,' which uses a precise verb and resource. It distinguishes itself from sibling tools like qt_attach and qt_build through its focus on session teardown.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit guidance on when to use purge=True: 'Use purge=True before rebuilding the library: otherwise the DLL file stays locked by the target process and qt_build fails.' It also explains the alternative behavior with purge=False, helping agents choose the right parameter value.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_detect_msvc_and_qtA
Detect MSVC (Visual Studio), MinGW toolchains, and Qt installations available for building.
IMPORTANT — this tool only discovers and displays what is found on this machine. It does NOT select an environment automatically.
After calling this tool, the AI MUST use AskUserQuestion to present
the found installations to the user and let them choose which compiler
and Qt to use with qt_build. For an MSVC Qt kit
(kit == "msvc") pass the VS vcvars_path + the kit's
qtenv_path with toolchain="msvc"; for a MinGW kit
(kit == "mingw") pass a MinGW toolchain's bin dir (from
mingw_toolchains) + the kit's qtenv_path (MinGW kits ship
qtenv2.bat too) with toolchain="mingw".
If detection finds nothing (or misses an installation the user knows
about), the user can type the paths manually.
Returns three lists: vs_installations, mingw_toolchains, and
qt_installations.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility for behavioral disclosure. It clearly states the tool is read-only ('only *discovers* and *displays*'), does not modify or select anything, and describes the edge-case behavior when nothing is found. This is comprehensive for a non-mutating detection tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than a single sentence, but every sentence serves a purpose: outlining the detection scope, emphasizing non-selection behavior, and giving concrete integration steps with qt_build. It is well-structured with an 'IMPORTANT' callout, though it could be slightly trimmed without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description fully explains the tool's role within the workflow, including return values (vs_installations, mingw_toolchains, qt_installations), how to use the results with qt_build, and the fallback for missing installations. Given the tool has no parameters and an output schema exists, the description covers all needed context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. There is no input schema to clarify, and the description appropriately focuses on output. It does not need to add parameter information, and the baseline is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool detects MSVC, MinGW toolchains, and Qt installations for building. It specifies the exact resources ('vs_installations', 'mingw_toolchains', 'qt_installations') and distinguishes itself from sibling tools like qt_build and qt_attach by focusing on discovery only.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: it is the discovery step before qt_build. It includes a mandatory instruction to use AskUserQuestion after calling, explains how to pass results to qt_build for both MSVC and MinGW kits, and tells the user to enter paths manually if detection fails. It also clarifies what the tool does NOT do ('does NOT select an environment automatically'), preventing misuse.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_find_elementA
Find UI elements matching a query in a session.
Query fields (all AND-ed): type (exact C++ class name),
type_inherits (superclass chain — C++ classes only; custom QML
components are generated classes like X_QMLTYPE_8 and do NOT match
a QML type name), text / text_contains (display text, CJK
supported), object_name, window_title /
window_title_contains, properties (all key-value pairs must
match), ancestor_id / window_id (scope limits), depth
("exact" = direct children, "shallow" = 2 levels, integer = that many,
"deep"/omitted = whole tree), limit (max matches). include_hidden
is a field INSIDE the query dict (default false) — hidden elements
are only matched when it is true.
Example: {"text_contains": "OK", "type_inherits": "QPushButton", "depth": "shallow"} or {"object_name": "searchBox", "limit": 5}.
Result: {"ok": true, "count": N, "elements": [{id, className, objectName, ...}]}; when nothing matches: {"ok": false, "message": "No matching element found"} — adjust the query or set include_hidden: true. No prior snapshot is needed; this call rebuilds the element map itself.
ID LIFECYCLE: the rebuild invalidates EVERY element_id/window_id from previous snapshots/finds. Use the returned ids immediately — if an operation reports "Element not found: id=N", the id is stale or the element was destroyed: re-run qt_find_element (or a snapshot) and retry with the fresh id; never reuse old ids.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and succeeds. It discloses the AND-ed query semantics, default hidden element handling, result shape for both success and no-match, and the critical side effect: every call invalidates all previous element/window IDs. The ID lifecycle warning about stale IDs and re-running is essential behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than typical but every sentence earns its place given the tool's complexity. It is well-structured with sections for query fields, examples, result format, and ID lifecycle. Minor redundancy (explaining result format when an output schema exists) keeps it from a 5, but it is dense and highly scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has a nested query object and an output schema, yet the description goes far beyond schema requirements. It covers query field semantics, depth/limit behavior, hidden element handling, error results, and ID invalidation—details an agent needs to invoke the tool correctly and handle failures gracefully. It is complete for this tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema treats 'query' as an opaque object with 0% property coverage, so the description must explain all valid keys. It does so comprehensively: type, type_inherits, text/text_contains, object_name, window_title variants, properties, ancestor/window_id, depth with exact values, limit, and include_hidden. It also gives two concrete examples, fully compensating for the schema's lack of detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Find UI elements matching a query in a session.' It clearly distinguishes this from sibling tools like qt_snapshot (snapshot creation) and qt_get_property (property inspection) by focusing on element discovery via a flexible query.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says 'No prior snapshot is needed; this call rebuilds the element map itself,' which clarifies when to use this instead of taking a snapshot first. It also instructs when to re-run the tool after stale IDs, referencing 'qt_find_element (or a snapshot)' as recovery options, effectively naming an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_focusA
Set focus on a UI element.
Best effort: QWidget focus works directly; some QML items may need an explicit focus request from within the app and report ok without changing focus.
| Name | Required | Description | Default |
|---|---|---|---|
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the best-effort nature and the specific QML limitation where the tool may report success without actually changing focus. This is valuable transparency about potential false positives, though it could mention more about error behavior or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is brief and front-loaded: two sentences that first state the purpose and then provide essential caveats. Every sentence adds value, with no redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the presence of an output schema, the description is reasonably complete. It covers the core action and a significant behavioral quirk. However, it lacks guidance on when to use this instead of other interaction tools and could clarify the role of session_id in the context of UI automation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage and the tool description does not explain the parameters. The names 'session_id' and 'element_id' are somewhat self-explanatory, but the description does not even hint at their roles, and there is no mention of required context or how element_id relates to the set-focus operation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: setting focus on a UI element. This is a specific verb+resource, and it distinguishes itself from sibling tools like qt_mouse_click or qt_set_property, which perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the use case (focusing a UI element) but does not explicitly compare to alternatives or state when not to use it. The caveat about QML items gives some context, but there is no clear directive on when to choose this over other methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_get_propertyA
Read a property from a UI element.
Read-only, no side effects on the target. Note: hidden, disabled and zero-size elements are rejected even for reads ("Element is not visible") — pass an id from a recent find/snapshot of a visible element.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses read-only behavior, no side effects, and the notable rejection of hidden, disabled, and zero-size elements—even for reads. This goes beyond the schema and helps prevent common errors.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with the primary purpose. The second sentence adds a crucial behavioral note without wordiness. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with an output schema, the description is adequately complete. It covers the principal behavior and the critical visibility constraint. It does not enumerate property names, but the existence of an output schema reduces the need to explain return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It adds some meaning by implying 'name' is the property to read and that 'element_id' must be from a recent find/snapshot, but it does not explicitly define session_id or enumerate valid property names. The parameter clarification is thin.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Read a property from a UI element') with a specific verb and resource. This distinguishes it from siblings like qt_set_property, which performs the opposite operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: it is read-only, and it specifies a prerequisite that the element id must come from a recent find/snapshot of a visible element. It does not explicitly name alternatives, but the sibling list and the read/write contrast imply when to use vs. qt_set_property.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_keyboard_inputA
Send keyboard input (typed text) to a UI element.
Real input: the text lands in whatever the target would receive — it can overwrite existing text and trigger real app behaviour. If element_id is 0 or does not resolve, the input goes to the widget that currently has focus (be careful: a stale id silently redirects the text). Widgets that rely on focus (QLineEdit etc.) may ignore input to an unfocused element — call qt_focus first when text does not land. Use qt_key_combo for shortcuts like Enter or Ctrl+C.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | ||
| modifiers | No | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so excellently. It discloses that input is real, can overwrite existing text, triggers real app behavior, may silently redirect on stale element_id, and that focus-dependent widgets may ignore input. This is rich, safety-relevant behavioral context beyond any schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with purpose, then adds necessary behavioral caveats. Every sentence earns its place: real-input warning, focus fallback, focus-dependent widgets, and the qt_key_combo alternative. No fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is largely complete given the output schema exists (so return values need not be described) and the core behavior is well covered. However, the missing semantics for modifiers and session_id create a real gap for an agent trying to invoke the tool correctly with all parameters. Still, the main usage path and key pitfalls are clearly covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains element_id semantics (0/unresolved → focused widget, stale id redirect) and text behavior (typed, can overwrite), but it does not describe the modifiers parameter (accepted values or combination rules) or session_id. This is partial compensation; two of four parameters remain semantically under-specified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Send keyboard input (typed text) to a UI element.' It clearly identifies the primary action and object, and distinguishes itself from the sibling qt_key_combo by explicitly carving out shortcuts for that tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides explicit usage guidance: use for typed text, use qt_key_combo for shortcuts like Enter or Ctrl+C, and call qt_focus first when text does not land. It also explains the fallback behavior when element_id is 0 or unresolved, which helps an agent decide when to invoke this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_key_comboA
Send a keyboard shortcut to an element, e.g. "Ctrl+C", "Ctrl+Shift+A".
Format: modifier names ("Ctrl", "Alt", "Shift", "Meta") joined with '+' followed by the key name ("C", "F5", "Enter", "Tab", "Escape", "Home", arrows, ...). element_id 0 targets the widget that currently has focus. Delivered as a real key press/release pair with the modifier state set — shortcuts can trigger real app actions (close window, save, submit).
| Name | Required | Description | Default |
|---|---|---|---|
| keys | Yes | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full behavioral disclosure. It explains that input is delivered as a real key press/release pair with modifier state set, and importantly warns that shortcuts can trigger real app actions like closing windows or saving—critical risk information. This goes beyond the basic purpose.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three efficient sentences: purpose, format, and behavioral nuance. No fluff or redundant information, and the most important details are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the essential aspects: purpose, key format, target element semantics, and real-action consequences. An output schema exists, so return values need not be described. The only minor gap is the undefined session_id, but overall the tool context is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema property descriptions are absent (0% coverage), so the description compensates by detailing the 'keys' format and the special meaning of element_id 0 for focus targeting. However, session_id is not explained, leaving a partial gap in parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Send a keyboard shortcut to an element', with concrete examples like 'Ctrl+C'. It uses a specific verb and resource, and the sibling tools (e.g., qt_keyboard_input) are visually distinguished by this shortcut-specific focus.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on how to use the tool, including the key format (modifiers joined with '+') and that element_id 0 targets the focused widget. It implies this is for shortcuts rather than general text input, but does not explicitly name alternatives or when-not-to-use conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_list_processesA
List running Qt processes that may be attachable.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description bears the full burden. It discloses the tool's purpose and the attachability filter, but it does not state whether the operation is read-only, what happens if no processes match, or any environmental prerequisites. The verb 'list' implies a non-destructive operation, providing some transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that leads with the verb 'List' and stays on point. Every word adds value, and there is no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters) and the presence of an output schema, the description is largely complete. It could mention that the process list is filtered by attachability, but that is included. It misses a note about potential prerequisites, but for a listing tool this is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description does not need to explain parameters, and it doesn't contradict the empty input schema. There is no additional semantic information to add.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'List' with a clear resource 'running Qt processes' and a qualifier 'that may be attachable', which distinguishes it from sibling tools like qt_list_sessions or qt_attach. It immediately communicates what the tool returns and its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage in the process of attaching to Qt processes (by mentioning 'attachable'), but it does not explicitly state when to use this tool or name alternative tools. No exclusions or prerequisites are mentioned, so it relies on the agent to infer from the sibling tool names.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_list_sessionsA
List all active sessions.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full burden. It indicates the tool lists only 'active' sessions, but does not disclose whether it is read-only, requires an attached target, or has any side effects. Minimal behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One short, front-loaded sentence with no waste. It states exactly what the tool does. Appropriate for a zero-parameter list operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no annotations and a simple tool, the description is minimally viable but lacks contextual nuances. It does not explain what 'sessions' means in this domain, nor mention related tools' relationships. Output schema exists, reducing some burden, but overall the description could be more informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are zero parameters, and the schema is fully covered (100%) with an empty object. The baseline for 0 params is 4, and the description adds no unnecessary parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List all active sessions.' uses a specific verb (List) and resource (active sessions), clearly distinguishing it from siblings like qt_list_processes. It is unambiguous and concise.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of when to list sessions instead of using other sibling tools, nor any exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_clickA
Send a mouse click to a UI element (direct delivery).
For plain QtWidgets (QPushButton, QLineEdit, ...) direct delivery
works and is the cheapest option. Use qt_mouse_click_region (element
center) or qt_mouse_click_at (exact coordinates) for QML custom
components (buttons, nav rows, list items) — those route through the
real Qt input pipeline with real hit testing and are far more
reliable. button: "left"/"right"/"middle" (anything else falls
back to left); modifiers: list of "Ctrl"/"Alt"/"Shift"/"Meta".
A result with ok:false means the element was rejected (hidden/
disabled/zero-size/stale id) — re-find and retry.
| Name | Required | Description | Default |
|---|---|---|---|
| button | No | left | |
| modifiers | No | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden and does disclose meaningful behavior: invalid button values fall back to 'left', invalid elements cause ok:false, and the reason for rejection (hidden/disabled/zero-size/stale id). It doesn't explicitly say whether the click performs real hit testing, though the contrast with the 'real Qt input pipeline' strongly implies it, so a small gap remains.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded: purpose first, then usage guidance, parameter semantics, and error handling. Every sentence adds value without being bloated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given an output schema exists and no annotations, the description covers the essential behavioral context: what the tool does, when to use alternatives, parameter semantics, and failure behavior. It is sufficient for an agent to select and invoke the tool confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It documents the allowed values for button and modifiers, including fallback behavior, which the schema does not provide. session_id and element_id are not elaborated, but their roles are clear from their names and the schema's required fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence states a specific action ('Send a mouse click to a UI element') with a qualifier ('direct delivery') that immediately differentiates it from sibling tools like qt_mouse_click_region and qt_mouse_click_at. The rest of the description reinforces this distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names alternative tools (qt_mouse_click_region, qt_mouse_click_at) and states when they should be used instead (QML custom components). It also explains the direct-delivery tradeoff ('cheapest option' for plain QtWidgets), giving clear selection guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_click_atA
Click at an exact window coordinate, exactly like a real mouse click at that position.
x/y are relative to the target window's client area (same space as screenshots, logical pixels — DPI is handled internally). The click goes through the real Qt input pipeline, so the hit test (scene graph for QML, widget tree for QtWidgets) determines what receives it — identical behavior to a human clicking there, including real consequences (menus, close buttons, destructive actions). window_id: id of a top-level window from the MOST RECENT snapshot/find (ids expire on refresh); 0 uses the session's first visible top-level window — pass it explicitly when the session has multiple windows.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| button | No | left | |
| modifiers | No | ||
| window_id | No | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and does so excellently. It discloses that the click goes through the real Qt pipeline, that the hit test determines the target, and that it may trigger destructive side effects like menus or close buttons. It also warns about window_id expiry, adding important temporal context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose, followed by precise, high-value details. Every sentence adds necessary information without fluff. The structure is easy to scan, with coordinate semantics and window_id warnings clearly separated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite lacking annotations, the description is remarkably complete for a complex tool. It covers coordinate space, pipeline behavior, risk of destructive actions, and window_id lifecycle. Given an output schema exists, return-value details are not needed, and the tool's usage context is fully understood.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It provides detailed semantics for x/y (client area, logical pixels, DPI handling) and window_id (recent snapshot, expiry, default behavior with 0). However, button and modifiers are left to schema defaults and are not explicitly explained, though their names and defaults make them reasonably clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool clicks at an exact window coordinate and emphasizes 'exactly like a real mouse click at that position.' This distinguishes it from sibling coordinate/region/element-based click tools, making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use this tool: when you need a pixel-exact click, with coordinates in the same space as screenshots and DPI handled internally. It does not explicitly name alternatives, but the contrast with element-based clicks (via the real Qt input pipeline) strongly implies the usage boundary.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_click_regionA
Click at the center of an element's on-screen region.
Unlike qt_mouse_click (which delivers straight to the element), this routes through the real Qt input pipeline with real hit testing: for a QML container (e.g. a Rectangle with a MouseArea inside) the scene graph hit test delivers the click to the MouseArea, exactly as a human click would.
| Name | Required | Description | Default |
|---|---|---|---|
| button | No | left | |
| modifiers | No | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the click routes through the real Qt input pipeline with real hit testing and provides a concrete example, offering meaningful behavioral context beyond a generic 'click'.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences that state the action and the key differentiator without waste. The description is front-loaded with the core action and the explanatory note is well-placed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple click operation with an output schema, the description covers the essential purpose and behavioral nuance. The missing parameter details are a minor gap, but the overall picture is clear for an agent to decide and invoke the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description should explain each parameter, but it only implicitly references element_id via 'element' and doesn't mention session_id, button, or modifiers. Button and modifiers have no description either in schema or description, leaving their semantics to inference from names alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses a specific verb ('Click at the center') and resource ('an element's on-screen region'), and explicitly contrasts with qt_mouse_click, distinguishing this tool's behavior from the sibling. The purpose is unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names qt_mouse_click as an alternative and explains the difference (real hit testing vs direct delivery), providing clear guidance on when to choose this tool. However, it doesn't mention other siblings like qt_mouse_click_at, which could also be relevant.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_dbl_clickB
Send a double-click to a UI element (direct delivery).
x/y are optional logical-pixel coordinates relative to the element's top-left corner; either both or neither (omitted = element center).
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| button | No | left | |
| modifiers | No | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'direct delivery', which vaguely implies a synthetic event, but does not mention side effects, prerequisites, cursor movement, or return behavior. This is insufficient for a tool that triggers a UI action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the primary action. The second sentence efficiently explains coordinate behavior. No filler or redundant content, but it could have used the brevity to include more safety or behavioral context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no annotations and a non-trivial parameter surface (6 params), the description is incomplete. It lacks context about session requirements, whether physical cursor movement occurs, and how button/modifiers interact with the double-click. The output schema existence does not compensate for these gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explains the x/y parameters in useful detail (relative coordinates, optional, center default), which adds value beyond the schema. However, it says nothing about 'button' (even though it has a default of 'left' and possible variants) or 'modifiers', and schema description coverage is 0%. These parameters remain semantically unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Send a double-click to a UI element'. This uses a specific verb and resource, and distinguishes the tool from single-click or other mouse tools. The 'direct delivery' note adds a mode distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on x/y coordinate semantics (optional, relative to element top-left, both or neither, omitted means center). However, it does not explicitly state when to use this tool over alternatives like qt_mouse_click or qt_mouse_press; usage context is only implied by the tool name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_moveA
Move the mouse pointer to an element-local position (no buttons).
Use between qt_mouse_press and qt_mouse_release to drag, or alone to hover. x/y are coordinates relative to the element's top-left corner.
| Name | Required | Description | Default |
|---|---|---|---|
| x | Yes | ||
| y | Yes | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It discloses that the tool moves the pointer without pressing buttons, that it can be used for dragging between press/release, and that coordinates are relative to the element's top-left corner. This adds meaningful context about how the tool behaves, though it does not mention potential error conditions or return value semantics, which are partially covered by the output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded. The first sentence states the core action and constraint ('Move the mouse pointer... no buttons'), while the second sentence provides usage context and coordinate semantics. Every sentence adds value without redundancy or fluff, making it highly concise and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, and the description provides the essential context: what it does, how to use it (drag/hover), and coordinate meaning. An output schema exists, so return values need not be described. It lacks mention of edge cases like invalid element IDs or out-of-bounds coordinates, but for a simple mouse-move tool, the description is largely complete. A small gap is the absence of any note about whether the move is instantaneous or animated, but this is not critical.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains x/y as 'coordinates relative to the element's top-left corner,' adding meaning to those parameters. However, session_id and element_id are not explicitly described, though their roles are inferable from the tool's name and context (session-based Qt automation). The description only partially compensates for the lack of schema descriptions, so it meets the baseline but does not fully document all parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Move the mouse pointer to an element-local position (no buttons).' This is a specific verb+resource combination that distinguishes it from sibling tools like qt_mouse_press, qt_mouse_release, and qt_mouse_click, which involve button actions. The phrase 'no buttons' explicitly differentiates it from click/press/release operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use between qt_mouse_press and qt_mouse_release to drag, or alone to hover.' This clearly states when to use the tool in a sequence and names sibling tools, giving context for the intended workflow. It does not explicitly exclude other use cases (e.g., clicking), but the 'no buttons' note and drag/hover guidance imply the appropriate context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_pressA
Press a mouse button on an element WITHOUT releasing it.
Pair with qt_mouse_release to split a click, or qt_mouse_move for a drag (press -> move -> release). x/y are optional logical-pixel coordinates relative to the element's top-left corner; either both or neither (omitted = element center). IMPORTANT: after a press you must release in the same session, or the target app stays in a pressed/ dragging state.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| button | No | left | |
| modifiers | No | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It does so exceptionally well, highlighting the stateful nature of the press (must release in the same session), the risk of leaving the app in a pressed/dragging state, and the coordinate interpretation (logical pixels relative to element top-left, omitted means center).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the core behavior, immediately followed by pairing guidance and a critical warning. Every sentence earns its place; no repetition or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity, the description covers all essential aspects: what it does, how it relates to siblings, coordinate behavior, and the critical statefulness warning. The presence of an output schema means return values need not be described, and the description is sufficiently complete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It meaningfully explains the x/y parameters (optional, both-or-neither, relative coordinates, default to center). It does not elaborate on 'button' or 'modifiers', though their names and schema defaults provide reasonable clues. This is strong compensation but not complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Press a mouse button on an element WITHOUT releasing it.' It clearly differentiates this from a complete click and references sibling tools like qt_mouse_release and qt_mouse_move, making the tool's unique purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: pair with qt_mouse_release to split a click, or use qt_mouse_move for a drag (press -> move -> release). It also explains the optional coordinate semantics and warns about the release requirement, giving the agent clear context for when and how to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_releaseA
Release a previously pressed mouse button on an element.
Completes a press/release pair (a click) or finishes a drag started with qt_mouse_press + qt_mouse_move. x/y are optional logical-pixel coordinates relative to the element's top-left corner; either both or neither (omitted = element center).
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| button | No | left | |
| modifiers | No | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It adds useful context about coordinate handling (optional x/y relative to top-left, default center) and typical usage scenarios. However, it does not disclose behavior in edge cases (e.g., no prior press), potential errors, or any side effects beyond the action itself, leaving gaps in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, with two sentences that efficiently convey purpose, usage context, and coordinate behavior. There is no redundant or filler content; every sentence adds necessary information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 6 parameters, no schema descriptions, and no annotations, so the description needs to cover essential context. It addresses the main use case and coordinate semantics, and since an output schema exists, return values need not be described. However, it omits details about parameter constraints (button values, modifier syntax) and edge-case behavior, making it less complete than ideal for a tool of moderate complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate by explaining parameter semantics. It does explain x/y coordinates and their optionality, which is valuable. However, it does not describe the 'button' parameter (values like left/right/middle) or 'modifiers' (how to specify key combinations), and does not explain the required session_id/element_id beyond what their names imply. Thus, parameter coverage is partial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('release') and resource ('previously pressed mouse button on an element'), clearly distinguishing it from sibling tools like qt_mouse_press and qt_mouse_click. It also explains the role in press/release pairs and drag sequences, leaving no ambiguity about the tool's function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool: to complete a press/release pair (a click) or finish a drag started with qt_mouse_press + qt_mouse_move. It provides useful sequencing context, though it does not explicitly list exclusions or alternatives beyond the implied counterparts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_mouse_wheelA
Scroll the mouse wheel over an element.
dx/dy are the wheel deltas in either pixel (pixel=True) or line
(default) units. x/y optionally position the wheel inside the
element (element-local logical px, both or neither — omitted = center).
Use this to scroll lists, tables, canvases and other scrollable
content.
| Name | Required | Description | Default |
|---|---|---|---|
| x | No | ||
| y | No | ||
| dx | No | ||
| dy | No | ||
| pixel | No | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It explains the units for dx/dy (pixel vs line), the optional x/y coordinate semantics (both or neither, default center), and the scrollable content scope. This goes beyond the schema by clarifying intended behavior, though it does not mention error handling or return values.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences: purpose, parameter details, and usage context. It is front-loaded with the action, and every sentence adds essential information without waste. The formatting with code literals improves readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (7 parameters) and the existence of an output schema, the description adequately covers the key nuances: wheel delta units, coordinate anchoring, and applicable UI components. It provides enough contextual richness for an agent to invoke the tool correctly without needing to guess semantics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It clearly explains dx/dy (wheel deltas, pixel or line units), pixel (unit selector), and x/y (optional positioning with 'both or neither' rule). The identity parameters (session_id, element_id) are self-explanatory given the tool context. This fully compensates for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise verb-object pair: 'Scroll the mouse wheel over an element.' This clearly distinguishes the tool from sibling tools like click, key input, and focus. It also implies the target resource (an element), making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The closing sentence, 'Use this to scroll lists, tables, canvases and other scrollable content,' provides direct guidance on when to apply the tool. It gives context but does not explicitly name alternative tools or exclusions, which would merit a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_prune_snapshotA
Occlusion-prune a saved snapshot.
snapshot_id is the value returned by the last qt_snapshot call of
this session. When to use: after a snapshot, when elements overlap
inside the same window and you need to know which one is really on
top / how much of a target is visible — e.g. before deciding which
element to click or whether a covered element matters.
Reads the saved snapshot JSON and computes what is actually visible:
elements fully covered by higher-z opaque elements are removed (their
still-visible descendants are reparented up), partially covered ones
get a visible_ratio field, fully hidden (opacity 0) elements are
dropped. Writes snapshot_<id>_pruned.json next to the original —
read the tree at the returned uri (same objIDs as the source
snapshot plus visible_ratio annotations). Result field
pruned = {"removed", "kept", "removed_ratio"}; window roots are
never removed.
The solver is a geometric heuristic (axis-aligned rects, per-window z-order with same-z tree order; widgets and QML rectangles/images occlude unless semi-transparent, transparent containers, text and custom QML components do not; a parent never occludes its own children). Occlusion is per top-level window on purpose: the agent must stay able to operate an app that the user has covered or minimised, so windows never occlude each other. The objIDs in the pruned file expire with the next snapshot/find refresh like any other id.
| Name | Required | Description | Default |
|---|---|---|---|
| session_id | Yes | ||
| snapshot_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and excels. It details the geometric heuristic, occlusion rules (e.g., 'widgets and QML rectangles/images occlude unless semi-transparent'), per-window behavior, file writing, and objID expiration. This provides deep transparency beyond basic read/write semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with a clear one-line purpose and is organized into coherent sections (when to use, behavior, solver details, output). It is longer than average but every sentence adds essential information about a complex algorithm. Slight verbosity around the heuristic details prevents a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and a nontrivial algorithm, the description is remarkably complete. It covers input requirements, output file, result field, edge cases (partial overlap, opacity, child reparenting), limitations (per-window occlusion), and lifecycle of objIDs. The presence of an output schema reduces the need to describe return values, allowing the description to focus on behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does explain the key parameter: 'snapshot_id is the value returned by the last qt_snapshot call of this session.' session_id is implied by context but not explicitly detailed. Since there are only two parameters and one is fully clarified, it earns above-baseline score but not a 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Occlusion-prune a saved snapshot.' It clearly distinguishes this tool from siblings like qt_snapshot by explaining it computes visibility and removes covered elements, which is a unique purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool: 'after a snapshot, when elements overlap inside the same window and you need to know which one is really on top / how much of a target is visible.' It gives concrete examples like deciding which element to click, but does not explicitly mention when not to use it or name alternative tools, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_screenshotA
Capture a screenshot of a UI element or the entire window.
element_id 0 captures the active (or first visible) top-level
window. The PNG is written to the file at the returned uri —
read that resource (qt-commander://sessions/.../screenshots/N.png) to
obtain the image bytes. On failure the result reports the error and
no image is written.
| Name | Required | Description | Default |
|---|---|---|---|
| element_id | No | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It reveals key behaviors: the PNG is written to a file at the returned URI, the resource must be read to obtain image bytes, and on failure no image is written and the result reports the error. This is valuable context beyond the schema, though it does not cover all potential side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is four concise sentences, front-loaded with the main action and followed by necessary operational details. No waste, and every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with two simple parameters and a clear output schema, the description is nearly complete. It covers the main action, how to retrieve the screenshot bytes, and error behavior. Minor gaps remain around session_id semantics, but the description is sufficient for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for parameter meanings. It explains element_id (0 captures the entire window) but does not clarify session_id. This partial compensation earns a middle score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool captures a screenshot of a UI element or the entire window, with a specific verb and resource. It does not explicitly differentiate from sibling tools like qt_snapshot, but the scope is unambiguous enough to understand the primary function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by explaining how element_id 0 captures the active/first visible top-level window, but it does not provide explicit when-to-use guidance or mention alternatives. It lacks exclusions or comparison with sibling tools, leaving usage context inferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_set_propertyA
Write a property value on a UI element.
DESTRUCTIVE: directly modifies the target application's state (may trigger property-notify signals, change geometry/visibility/business state) with no undo. Prefer real user input (clicks/typing) to simulate user actions; use this only when a direct API write is intended.
value is first parsed as JSON: "true" -> bool, "123" -> int,
""hello"" -> string. Unparseable text is sent as a plain string —
pass quoted strings explicitly when a string is intended.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| value | Yes | ||
| element_id | Yes | ||
| session_id | Yes |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that the tool is 'DESTRUCTIVE,' directly modifies application state, may trigger property-notify signals, change geometry/visibility/business state, and has no undo. It also details value parsing behavior, adding critical behavioral context beyond the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-line purpose, a clear warning block, and a focused value-parsing note. Every sentence adds necessary information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a destructive write operation with no annotations and a 0%-coverage schema, the description covers the key aspects: what it does, when it should be used, its side effects, and value semantics. The output schema exists, so return-value details are not needed. It is sufficiently complete for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It thoroughly explains the `value` parameter with JSON parsing rules ('"true" -> bool, "123" -> int'), but does not explicitly describe session_id, element_id, or name. However, their meanings are reasonably inferable from the purpose statement and parameter names, making this a partial compensation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with 'Write a property value on a UI element,' which clearly names the action and resource. It is distinct from sibling tools like qt_get_property (reading) and qt_call_method (calling methods), making its purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'Prefer real user input (clicks/typing) to simulate user actions; use this only when a direct API write is intended.' This provides clear when-to-use and when-not-to-use guidance, contrasting with alternative input methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
qt_snapshotA
Capture the UI element tree of the session's windows.
The tree is NOT in this result — it is written to the file exposed at
the returned uri (resource qt-commander://sessions/.../snapshots/ snapshot_N.json); read that resource to inspect the tree. The result
carries only session_id / snapshot_id / uri. Node fields: className,
objID, objectName, rect (window-local logical px), global_rect (screen
coords), z_order, visible/enabled/opacity/color_alpha/clip, text,
topLevelId, windowTitle, properties (per detail).
detail — property tier per node: "core" (first-class fields only,
no properties), "extended" (DEFAULT; common interaction-state
properties like text/checked/value), "full" (every Q_PROPERTY; slow on
large UIs).
include_hidden (default False) — hidden elements are pruned
together with their whole subtree (hidden tabs, unopened dialogs).
Set True to include them; note hidden elements are still rejected by
click/input/property tools.
max_depth — levels of children: 0 = roots only, 1 = roots +
direct children, -1 = entire tree (default 1).
prop_depth — levels of QObject property expansion (0 = none,
-1 = unlimited).
root_id — 0 = all top-level windows; >0 = subtree of an element
from the MOST RECENT snapshot/find. Ids expire on every refresh: if
root_id no longer resolves, the tool silently falls back to all
top-level windows — pass it only when you know it is fresh.
ID LIFECYCLE: this call rebuilds the element map, so every element_id and window_id from previous snapshots/finds becomes invalid. The snapshot runs on the target process's GUI thread (brief UI freeze); if the target is busy, calls can time out after ~30s.
| Name | Required | Description | Default |
|---|---|---|---|
| detail | No | extended | |
| root_id | No | ||
| max_depth | No | ||
| prop_depth | No | ||
| session_id | Yes | ||
| include_hidden | No |
Output Schema
| Name | Required | Description |
|---|---|---|
| result | Yes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden, and it excels: it explicitly states that the tree is written to a file and not in the result, describes the ID lifecycle (previous IDs become invalid), warns about GUI thread freezes and 30s timeouts, and explains the fallback behavior for stale root_id. These are non-obvious, important behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured: a lead sentence, a note about the output location, parameter explanations, and a clear ID LIFECYCLE section. Every sentence carries essential information, though the density makes it somewhat heavy; still, it earns a high score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 params, resource output, ID lifecycle, GUI thread behavior), the description covers all critical aspects: what is returned, where to read the tree, node fields, parameter semantics, freshness constraints, and side effects. An output schema exists, so not detailing return values is acceptable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate, and it does comprehensively. Every parameter is explained with meaning, default, and allowed values (e.g., detail as 'core/extended/full', max_depth as levels, prop_depth as QObject property expansion). This adds far more value than the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific verb+resource: 'Capture the UI element tree of the session's windows.' It clearly distinguishes this tool from siblings like qt_screenshot (visual capture) and qt_find_element (search within a tree).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides detailed guidance on parameter usage, including the semantics of detail, include_hidden, max_depth, prop_depth, and root_id. It notes that IDs expire on refresh and that root_id should only be passed when fresh, and warns about hidden elements being rejected by other tools. However, it does not explicitly compare against alternative tools like qt_find_element, so a 4 is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
25 tool updates
v0.1.0- First observed
qt_attach - First observed
qt_build - First observed
qt_call_method - First observed
qt_detach - First observed
qt_detect_msvc_and_qt - First observed
qt_find_element - First observed
qt_focus - First observed
qt_get_property - First observed
qt_key_combo - First observed
qt_keyboard_input - First observed
qt_list_processes - First observed
qt_list_sessions - First observed
qt_mouse_click - First observed
qt_mouse_click_at - First observed
qt_mouse_click_region - First observed
qt_mouse_context_menu - First observed
qt_mouse_dbl_click - First observed
qt_mouse_move - First observed
qt_mouse_press - First observed
qt_mouse_release - First observed
qt_mouse_wheel - First observed
qt_prune_snapshot - First observed
qt_screenshot - First observed
qt_set_property - First observed
qt_snapshot
TDQS
Scored across 25 tools
Most tools target distinct resources and actions, but the three mouse click variants (qt_mouse_click, qt_mouse_click_region, qt_mouse_click_at) are closely related and could be confused. Their descriptions clearly explain the differences in delivery mechanism and use cases, so ambiguity is mostly resolved.
All tool names follow a consistent qt_verb_noun pattern with lowercase underscores (e.g., list_processes, get_property, mouse_click). Minor abbreviations like 'dbl' in qt_mouse_dbl_click are negligible and do not disrupt the overall predictability.
With 25 tools, the server is on the heavier side, but each tool maps to a distinct step in Qt UI automation—from environment detection and build to attach, inspection, and input simulation. The granularity is justified by the complexity of the domain.
The tool set covers the entire lifecycle: detect environment, build, attach, snapshot/find elements, read/write properties, call methods, simulate mouse and keyboard input, and detach. No obvious gaps prevent agents from accomplishing UI automation tasks.
Maintenance
Related MCP Connectors
MCP server for building and testing AI agents with multi-model experimentation and insights.
MCP server for visual regression testing: triage a PR's UI diffs from your coding agent.
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Melaya is a remote MCP server. It gives an assistant hands on your own Android phone and browser: it reads the screen through the accessibility tree, then taps, types and navigates inside the apps and sites you allow-list, with no per-app API. It also builds, schedules and runs agent pipelines across 6k+ connected tools. OAuth 2.1, nothing to install.
Related MCP Servers
- AlicenseAqualityDmaintenanceAn MCP server for inspecting, debugging, and interacting with PySide6 desktop applications. It enables AI agents to capture UI snapshots, read widget properties, and perform actions like clicking buttons or typing text.222Apache 2.0
- FlicenseNot gradedqualityBmaintenanceMCP server that enables agents to control native GUI apps by snapshotting accessibility trees and performing actions like click, type, and scroll without bringing the app to the foreground.-
- AlicenseNot gradedqualityCmaintenanceGives AI agents and MCP clients direct control over native desktop apps, Chrome/Electron browsers, and Android devices with screenshots, OCR, accessibility-based element lookup, input simulation, window management, CDP, and ADB in one local server.130MIT
- AlicenseAqualityCmaintenanceAn MCP server for capturing screenshots of Qt/desktop windows and performing filesystem operations, enabling AI clients to inspect and modify project files.15MIT