MCP Server By TestMu AI
Server Details
Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers.
The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites:
HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps.
Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client.
Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.
- Status
- Healthy
- Uptime
- 24.6% over 66 days
- OAuth
- Works in Glama
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-03-26
- URL
TDQS
Scored across 88 tools
The 88-tool surface spans many LambdaTest products and includes several overlapping retrieval paths (e.g., getAutomationTest* vs getSessionV2* logs, multiple granular HyperExecute getters vs analyzeHyperExecuteJob). Descriptions do explain boundaries and v1/v2 distinctions, but the sheer number of similar-sounding tools leaves meaningful misselection risk.
Most tools use camelCase verb_noun (getBuild, deleteSession, listTunnels), but entire groups use snake_case with prefixes (smartui_*, tm_*) and one tool is upload_app. This mixes camelCase and snake_case conventions across the set, making naming unpredictable despite some group-level consistency.
88 tools is far beyond the typical 3–15 range and even beyond the 25+ heavy threshold. While the server covers a broad enterprise platform, this count is an extreme mismatch for a single MCP tool set and creates a high cognitive load for an agent.
Coverage is broad across web/mobile automation, HyperExecute, SmartUI, Test Manager, accessibility, and tunnels, with many create/read/update/delete operations present. However, notable gaps exist: Test Manager deletion is explicitly unavailable for projects, folders, test cases, and milestones; there is no stop/abort for HyperExecute jobs; and tunnel creation is not directly supported.
Available Tools
88 toolsanalyzeAppViaTunnelAnalyze App via TunnelBRead-onlyInspect
Analyzes a locally running app for accessibility via LambdaTest tunnel.
| Name | Required | Description | Default |
|---|---|---|---|
| port | Yes | Port where the app is running | |
| tunnelId | Yes | Tunnel ID | |
| tunnelName | Yes | Tunnel name |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, openWorldHint=false, and destructiveHint=false, so safety is covered. The description adds that the app must be local and tunneled, but says nothing about whether the analysis is synchronous, whether a status poll (e.g. checkAppProcessingStatus) is needed, or what triggers the scan.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single tight sentence with the key scoping qualifier ('locally running', 'via tunnel') front-loaded. Nothing is wasted, though it is arguably too terse to carry the workflow context an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should clarify what the call yields (e.g. a scan/report id) and whether further steps are required. It leaves the completion path implicit despite the presence of related siblings like checkAppProcessingStatus and getAccessibilityScanReport.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and all three parameters (port, tunnelId, tunnelName) are self-explanatory and documented in the schema. The description adds no extra meaning beyond what the schema provides, which is the expected baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Analyzes), the resource (a locally running app), the goal (accessibility), and the transport (via LambdaTest tunnel). It is clear about what it does, though it does not explicitly contrast itself with nearby siblings such as buildLocalAppForAnalysis or getAccessibilityReport.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'locally running app ... via LambdaTest tunnel' implies the preconditions (an app served on a local port reachable through a tunnel), but there is no explicit when-to-use vs when-not guidance or named alternative for already-uploaded apps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
analyzeHyperExecuteJobAnalyze HyperExecute JobARead-onlyInspect
Perform a full analysis of a HyperExecute job: fetches job info, sessions, traverses all tasks → scenario stages → console logs, and returns a structured report with test results, per-scenario logs, and pass/fail statistics. Use this for questions like "which tests failed and why" or "show me the console logs for job X".
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The HyperExecute job ID to analyze |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is lower. The description still adds useful behavioral context: it discloses that the tool makes a deep recursive traversal of all tasks and stages and returns an aggregated structured report rather than a single resource, implying heavier cost/latency than the granular siblings.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no waste. The core capability is front-loaded in the first sentence and the second sentence supplies the routing examples, so the reader gets purpose before use cases.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by naming what the report contains (test results, per-scenario logs, pass/fail statistics). Combined with annotations covering safety and a fully documented single required parameter, an agent has everything needed to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is a single parameter with 100% schema description coverage, so the schema already documents jobId fully. The description adds only marginal meaning (that the id is the job to analyze), which is the expected baseline when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (analyze/fetch/traverse) and resource (HyperExecute job) and enumerates exactly what the deep traversal covers: job info, sessions, tasks → scenario stages → console logs, plus the returned report contents. This scope description makes it distinguishable from the granular per-resource siblings like getHyperExecuteJobInfo or getHyperExecuteStageLogs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives use cases with two concrete example questions ('which tests failed and why', 'show me the console logs for job X'), which tells the agent when to pick this aggregate tool. It stops short of naming alternatives such as answerHyperExecuteQuery or the individual getter tools, so there is no when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
answerHyperExecuteQueryAnswer HyperExecute QuestionBRead-onlyInspect
Answers questions about HyperExecute features, configuration, usage, troubleshooting, and best practices by searching through documentation.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | Question about HyperExecute |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety and closed-domain profile is covered. The description adds that answers are sourced from documentation (not live job state), which is useful, but it says nothing about grounding, citation behavior, or answer completeness.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence that spends its words on the tool's purpose and mechanism with no filler. The enumerated topics are slightly list-heavy but each earns its place by sketching the question domain.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and a trivial input schema, the description covers the essentials: what it answers and that it searches documentation. It stops short of describing the answer format, source citations, or limitations, which would help an agent know what to expect back.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is a single parameter with 100% schema description coverage, so the schema already defines 'query' as a question about HyperExecute. The description adds no syntax, phrasing, or scope detail beyond that, making the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('answers questions about HyperExecute features...') and names the mechanism ('searching through documentation'). It clearly delineates the topical scope, though it does not explicitly distinguish itself from any named sibling tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the topical scope (HyperExecute questions), but there is no explicit when-to-use/when-not guidance or pointer to alternatives such as analyzeHyperExecuteJob for job-specific data rather than general documentation questions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
automationQueryAutomation QueryARead-onlyInspect
Read-only query of LambdaTest automation resources: list builds, get one build, list the sessions in a build, fetch test details, and retrieve test logs (command, network, console, selenium, instrumentation). Every combination works on platform "web" or "app": build list or get, session list or get, test get, and logs get with a logType. List results carry a pagination object; trust it, not result_set.total, which also counts older builds the list no longer returns. logType "instrumentation" returns XCUI and Espresso test logs. Renaming, stopping and deleting builds are separate tools: updateBuild, stopBuild and deleteBuild.
| Name | Required | Description | Default |
|---|---|---|---|
| sort | No | Sort order for build listing (e.g. "asc", "desc") | |
| limit | No | Number of results to return, 1 to 100 | |
| action | Yes | The read to perform: list, or get one | |
| offset | No | Offset for pagination; pass the next offset from a previous response | |
| status | No | Filter builds by status (e.g. "running", "passed", "failed") | |
| testId | No | Test ID — required for test get and logs get | |
| todate | No | Filter builds to this date (YYYY-MM-DD, not before fromdate) | |
| buildId | No | Build ID — required for build get, optional filter for session list | |
| logType | No | Log type — required when resource is "logs" | |
| fromdate | No | Filter builds from this date (YYYY-MM-DD, a real calendar date) | |
| platform | Yes | The platform type: "web" for browser automation, "app" for mobile app automation | |
| resource | Yes | The resource type to query | |
| sessionId | No | Session ID: required for session get and for app platform log retrieval. Session get also accepts the test ID instead. | |
| pageNumber | No | Page number for command log pagination, from 1 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds meaningful behavioral context beyond annotations: the pagination-object caveat versus result_set.total, and the fact that logType 'instrumentation' returns XCUI and Espresso logs. It omits auth constraints and rate limits, so it is strong but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core purpose, then lists capabilities and caveats in a compact, information-dense paragraph. Every sentence earns its place: the resource enumeration, platform applicability, pagination warning, and sibling routing all add distinct value with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 14 parameters, a multi-action dispatcher design, and no output schema, the description covers the essential call semantics: resource/action combinations, platform scope, logType behavior, and pagination handling. It does not describe return shapes for get/list operations or error behavior, but for a read-only query tool with rich schema coverage this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents every parameter, including required combinations and logType enums. The description reinforces useful relational semantics—every resource/action combination works on web or app, and logs require a logType—but adds little syntax or format detail beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource family: 'Read-only query of LambdaTest automation resources', then enumerates the exact reads it covers (builds, sessions, tests, logs). It distinguishes itself from mutation siblings by explicitly naming updateBuild, stopBuild, and deleteBuild, so an agent can tell it apart without inspecting schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear when-to-use context: read-only automation queries across web and app, with explicit exclusions for build mutations (updateBuild, stopBuild, deleteBuild). However, it does not distinguish itself from specialized log-retrieval siblings like getAutomationTestCommandLogs or getSessionV2CommandLogs, leaving overlap unresolved.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
buildLocalAppForAnalysisBuild and Analyze AppARead-onlyInspect
Returns step-by-step instructions to build and serve a local web app and open a LambdaTest tunnel to it, so its accessibility can then be scanned with analyzeAppViaTunnel. It does not build, run or scan anything itself, and never includes the access key.
| Name | Required | Description | Default |
|---|---|---|---|
| port | Yes | Port to host the app locally | |
| codebasePath | Yes | Absolute path to the codebase directory | |
| customBuildCommand | No | Custom build command (optional, e.g., npm run build) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds genuinely new behavioral context: it performs no build/run/scan side effects itself, and it guarantees the access key is never emitted — a meaningful security disclosure for a tool whose output is instructions containing tunnel setup steps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler; the core purpose is front-loaded and the second sentence adds only the two constraints that matter (no execution, no access key). Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must convey what is returned, and it does — step-by-step build/serve/tunnel instructions. It also covers the relationship to the downstream scan tool and the scope limits, which is everything an agent needs to call and use it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and all three parameters (port, codebasePath, customBuildCommand) carry their own descriptions, so the schema does the heavy lifting. The description adds no parameter-level detail such as port/range constraints or how customBuildCommand interacts with the default build — baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and output ('Returns step-by-step instructions to build and serve a local web app and open a LambdaTest tunnel'), and explicitly disclaims doing the building, running or scanning itself. It also names the sibling tool (analyzeAppViaTunnel) that consumes the result, so it is distinguishable from siblings without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes the workflow ordering explicit: this tool produces instructions so accessibility 'can then be scanned with analyzeAppViaTunnel', which tells the agent where it sits relative to that sibling. It doesn't state when to prefer this over, say, analyzeHyperExecuteJob or direct uploads, but the sequencing context is clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
checkAppProcessingStatusCheck App Processing Status (Real Device)AIdempotentInspect
Check whether an uploaded app has been processed and is ready for features like image injection, network logs, and screenshot unblock on real devices. The LambdaTest API has no read-only status call: this one starts or refreshes the processing for the requested features, so the app's last-updated time changes. An app id that is not among your uploaded apps returns an error.
| Name | Required | Description | Default |
|---|---|---|---|
| appId | Yes | The app ID to check | |
| networkLogsEnabled | No | Check network logs support | |
| imageInjectionEnabled | No | Check image injection support | |
| screenshotUnblockEnabled | No | Check screenshot unblock support |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds substantial behavior beyond the annotations: the API exposes no read-only status call, so this call starts or refreshes processing, mutating the app's last-updated timestamp, and an unknown app id returns an error. This directly resolves the trap of reading a 'check' tool as a safe read (annotations do correctly declare readOnlyHint=false).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no redundancy, and the mutated-state warning is front-and-center right after the purpose. Every sentence carries distinct information (readiness semantics, mutation side effect, error behavior).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description should ideally hint at what the returned status looks like (e.g., pending/processed/failed and per-feature readiness), which is the one gap for a 'check status' tool. Otherwise it covers purpose, side effects, and error behavior adequately for a 4-param mutation-adjacent tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all four parameters, making 3 the baseline. The description earns above baseline by explaining that the boolean flags correspond to 'the requested features' whose processing is started/refreshed, tying the flags to an observable side effect rather than just 'check X support'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('check whether an uploaded app has been processed') and scopes it to real devices and the features it gates (image injection, network logs, screenshot unblock). It cannot be confused with listUploadedApps or deleteUploadedApps, though it never names a sibling tool to route away from.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The implied usage is clear enough — call it to confirm an uploaded app is ready before relying on injection/network-log/screenshot-unblock features. However, it never states when NOT to call it, names no alternative, and gives no precondition such as 'requires the app to be uploaded first'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deleteBuildDelete Automation BuildADestructiveIdempotentInspect
Permanently deletes a LambdaTest web automation build and its test results. Cannot be undone.
| Name | Required | Description | Default |
|---|---|---|---|
| buildId | Yes | ID of the build to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true, so the safety profile is known. The description adds real value beyond that: it discloses cascading deletion of associated test results and irreversibility, which the annotations do not express.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two terse sentences with the destructive scope front-loaded and the irreversibility warning immediately after. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter mutation tool with full annotation coverage and no output schema, the description covers the essential unknowns: permanence and cascade scope. Missing only edge-case context such as permission requirements or behavior when the build does not exist.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Single buildId parameter with 100% schema description coverage, so the baseline is 3. The description adds no format, type, or sourcing detail for buildId beyond what the schema already states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (permanently deletes) and resource (LambdaTest web automation build plus its test results). This clearly separates it from stopBuild and updateBuild in the sibling list, which act on the same resource with different semantics.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to delete versus stop (stopBuild) or update (updateBuild) a build, and no mention of prerequisites or authorization. The 'Cannot be undone' warning signals severity but is not a usage routing rule.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deletePrerunFileDelete Prerun FileBDestructiveInspect
Delete an uploaded prerun file by stored LambdaTest file path.
| Name | Required | Description | Default |
|---|---|---|---|
| filePath | No | Stored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>) | |
| file_path | No | Stored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, readOnlyHint=false, and openWorldHint=false, so the safety profile is covered. The description adds only the path-based identity mechanism and does not discuss irreversibility, required permissions, or what happens to a file still referenced by a pre-run/post-run configuration.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler, appropriately sized for a simple delete operation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool the description covers the bare minimum: what is deleted and by what key. With no output schema and zero required parameters, it omits return/confirmation behavior and any caveat about the duplicate path parameter, so it is adequate but thin.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. However, the schema exposes two near-identical parameters (filePath and file_path) and the description does not clarify which one is authoritative or that the value must originate from uploadPrerunFile, leaving a real ambiguity unaddressed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Delete') and resource ('an uploaded prerun file') plus the identifier mechanism ('stored LambdaTest file path'). It is clear what the tool does, but it does not distinguish itself from similar siblings such as deleteUserFile or deleteUploadedApps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no preconditions, and no mention of alternatives despite several delete-oriented siblings (deleteUserFile, deleteUploadedApps, deleteBuild, deleteSession) that an agent must choose between. The agent is left to infer the distinction entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deleteSessionDelete SessionADestructiveInspect
Permanently delete a specific automation test session from LambdaTest by session ID.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The session ID to delete |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, readOnlyHint=false and openWorldHint=false, so the destructive nature is known. 'Permanently' reinforces irreversibility, which is a small addition, but nothing is said about permissions, side effects, or whether the parent job is affected.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence naming verb, resource, qualifier and identifier. Zero waste, no repetition of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter destructive tool whose annotations already cover the safety profile and with no output schema, the description is essentially sufficient. Only the sibling-choosing guidance is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single parameter has its own description ('The session ID to delete'), so the schema carries the semantics. The description adds no format or sourcing detail beyond it; baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (permanently delete), resource (automation test session), and scope (by session ID) on LambdaTest. It does not explicitly name the near-sibling stopSession, but 'permanently' implicitly distinguishes it from a mere stop.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied: delete a session you no longer need. However, it never states when to prefer stopSession over deleteSession, or any prerequisites (ownership, active vs completed session). No explicit when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deleteUploadedAppsDelete Uploaded AppsADestructiveInspect
Delete one or more uploaded apps from LambdaTest by their app IDs.
| Name | Required | Description | Default |
|---|---|---|---|
| appIds | Yes | Array of app IDs to delete (at least one) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, readOnlyHint=false, and openWorldHint=false, so the safety profile is covered externally. The description confirms deletion but adds no depth beyond that — no mention of irreversibility, whether apps in active use are blocked, or any confirmation requirement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler; the verb, resource, and key are all in the first clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter delete operation whose safety profile is carried by annotations, the description is largely sufficient and no output schema is needed. It stops short of noting whether deletion is permanent or has side effects on linked test runs, which would round it out.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the single appIds parameter fully documented as an array with minItems 1. The description's 'by their app IDs' and 'one or more' merely restate the schema, adding no format or constraint detail beyond it. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (delete), resource (uploaded apps), scope (one or more), and the key used (app IDs). An agent can distinguish this from sibling deletion tools like deleteBuild, deleteSession, and deleteUserFile without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage through 'delete one or more uploaded apps' but never states when to use this over listUploadedApps or the upload tools, nor any preconditions. No when/when-not guidance or named alternatives are present.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
deleteUserFileDelete User FileBDestructiveInspect
Delete an uploaded user file by stored LambdaTest file path.
| Name | Required | Description | Default |
|---|---|---|---|
| filePath | No | Key of the stored user file, as uploadUserFile returns it (usually the file name) | |
| file_path | No | Key of the stored user file, as uploadUserFile returns it (usually the file name) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and readOnlyHint=false, so the agent knows this is an irreversible write; the description correctly does not contradict that. However it adds almost nothing on top of the annotations: no statement that deletion is permanent, no auth/permission requirement, no behavior when the path does not exist. With annotations carrying the safety profile, this is a minimum-viable 3.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with the verb and the identifying key in the first few words, and zero filler or restatement of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive delete with no output schema, the description covers the essential purpose but leaves the notable ambiguity in the schema unresolved: filePath and file_path are both optional and identically described, and the description does not say which one is canonical or whether either is required. It is adequate but not complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters are already documented as 'Key of the stored user file, as uploadUserFile returns it'. The description repeats the same idea ('by stored LambdaTest file path') without adding syntax, format, or precedence guidance, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb (Delete) and resource (an uploaded user file) qualified by an identifier source (stored LambdaTest file path), which is enough to separate it from deleteBuild, deleteSession, and deletePrerunFile. It does not explicitly name those siblings or state what distinguishes a 'user file' from a 'prerun file', so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no mention of the neighboring delete tools (deletePrerunFile, deleteUploadedApps) that an agent must choose between. The only implicit cue is the word 'uploaded', which does not constitute routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
downloadJobArtifactDownload Job ArtifactARead-onlyInspect
Get an authenticated download URL (plus size/status metadata) for a specific HyperExecute job artifact. Requires artefactName (from getHyperExecuteJobArtifacts); optionally scope to a task with taskId. Returns a URL to fetch with your LambdaTest credentials rather than the raw file bytes.
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The HyperExecute job ID whose artifact to download | |
| taskId | No | Optional: scope the artifact to a specific task ID | |
| artefactName | Yes | Name of the artifact to download (as returned by getHyperExecuteJobArtifacts) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false). The description adds genuinely useful behavior beyond them: it returns a URL rather than bytes, and that URL must be fetched with LambdaTest credentials, which is critical for correct downstream use.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences: what it returns, the prerequisites, and the key behavioral caveat. Nothing is redundant and the most important fact (URL, not bytes) is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the return-value burden and does so by naming the URL plus size/status metadata. Auth requirements, the required input source, and optional scoping are all covered for a simple three-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all three parameters are already documented, including that artefactName comes from getHyperExecuteJobArtifacts. The description restates that origin but adds no new format, validation, or scoping semantics. Baseline 3 applies when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Get an authenticated download URL ... for a specific HyperExecute job artifact') and clarifies the return contract ('URL to fetch ... rather than the raw file bytes'). This cleanly separates it from getHyperExecuteJobArtifacts (which lists names) and getJobArtifactDetail.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It names the prerequisite source for artefactName ('from getHyperExecuteJobArtifacts') and explains when to use the optional taskId scope. It gives clear context for use but does not state explicit exclusions or contrast with the closest sibling beyond that prerequisite mention.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
generateHyperExecuteYAMLGenerate HyperExecute YAMLARead-onlyInspect
Generates a HyperExecute YAML configuration for running tests on LambdaTest's HyperExecute platform.
REQUIRED fields: project (language, framework) and testing (testDiscovery.command, testRunnerCommand) All other fields are OPTIONAL.
Supports multiple languages and frameworks:
JavaScript/TypeScript: Playwright, Cypress, WebDriverIO, Puppeteer, Taiko
Python: Pytest, Behave, Robot Framework
Java: TestNG, JUnit, Cucumber
Ruby: RSpec, Capybara
C#: NUnit, xUnit, MSTest
Mobile (ONLY use appTesting for these): XCUI (iOS), Espresso (Android), Appium
Features:
Smart framework defaults for common configurations
Input validation with security checks
Caching and artifact upload configuration
Git integration via sourcePayload
Mobile app testing support (via optional appTesting field)
The appTesting field applies to mobile app testing only; web automation (Selenium, Playwright, Cypress, etc.) leaves it out.
| Name | Required | Description | Default |
|---|---|---|---|
| setup | No | Setup configuration (optional) | |
| project | Yes | Project configuration | |
| testing | Yes | Test configuration | |
| advanced | No | Advanced overrides (escape hatch for any HyperExecute field) | |
| jobLabel | No | Labels for the job | |
| artifacts | No | Artifact configuration (optional) | |
| execution | No | Execution settings (optional) | |
| appTesting | No | Mobile app testing configuration (XCUI, Espresso or Appium). Web automation (Selenium, Playwright, Cypress, etc.) leaves it out. | |
| sourcePayload | No | Git integration: fetches the test code from a remote Git repository. Only needed when the code is not already present locally; local code (the usual case) leaves it out. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds modest behavioral context: smart framework defaults, input validation with security checks, and which fields are optional. It does not clarify whether output is returned as text or written to a file, so it only partly exceeds the annotation baseline.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose and required fields are front-loaded, and the bulleted framework list is scannable. There is some redundancy (appTesting is explained in the features list and again in a closing paragraph) and the feature bullets lean promotional, but overall it is well organized and appropriately sized for a 9-parameter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex, deeply nested tool with no output schema, the description covers required inputs, supported frameworks, and the mobile-vs-web branch well. Its one notable gap is the return value – it never says whether the tool returns YAML text, a file path, or a validated object – which matters since there is no output schema to fill that role.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents every parameter and its nested fields. The description restates the required fields (project, testing) and the mobile/web distinction, which mirrors rather than extends the schema. Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Generates a HyperExecute YAML configuration for running tests on LambdaTest's HyperExecute platform'), which is unambiguous and distinct from every sibling (none of which produce configuration artifacts). It also names the concrete required fields, so an agent immediately knows the minimum viable call.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear conditional guidance: mobile frameworks 'ONLY use appTesting for these', and web automation 'leaves it out'. It also flags which fields are required vs optional. However, it never states when to prefer this tool over alternatives or any preconditions (e.g., that this only generates config, not runs tests), so it stops short of explicit when/when-not routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAccessibilityReportGet Accessibility ReportAInspect
Runs an accessibility scan on the given URLs and returns the issues found. Pass urls to start a scan, or testId to re-read a scan that already ran; exactly one of the two. Results arrive in issueDetails: allIssues (most-severe-first, one page at a time), rules (name and reference link per rule id, listed once), pages (every scanned page with its score and issue count), failedRules (per-rule totals for the whole scan) and pagination. Each issue carries its own howToFix and criteria, measured for that element - two issues sharing a rule can have different causes and fixes. While pagination.nextOffset is not null the results are incomplete: call again with the testId from the response plus that offset, keeping the same filters, until it is null. Page with testId, never by passing urls again - that starts a new scan, costs another run, and can return a different set of issues whose offsets do not line up. a11yReportLink is the dashboard URL and needs a login; shareableLink is public and is only returned by the original scan.
| Name | Required | Description | Default |
|---|---|---|---|
| urls | No | List of URLs to scan. Starts a new scan. Provide either urls or testId, not both. | |
| limit | No | Maximum issues to return in this response (default 50, max 200). | |
| offset | No | Index to start from within the filtered, severity-sorted issue list. Pass the nextOffset from a previous response, together with that response's testId, to page through results. | |
| testId | No | Re-read an existing scan by its testId (as returned in a previous response) instead of running a new one. Use this to page through results with offset. | |
| pageUrl | No | Only return issues found on this scanned page. Must exactly match one of the URLs in issueDetails.pages. | |
| severity | No | Only return issues of this impact level. Omit to get all severities. | |
| include_rules | No | Return the full rule catalog (about 11 KB) instead of the default ruleCatalog summary of counts per category and disabled rule ids. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations (readOnlyHint=false, openWorldHint=true, destructiveHint=false) signal a non-read compute action with external effects; the description adds the crucial cost implication ("costs another run"), the auth distinction between a11yReportLink (needs login) and shareableLink (public, scan-only), and the pagination completion contract. This is substantial context the structured fields do not carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Purpose is front-loaded, then output shape, then pagination workflow, then link caveats. Despite its length, every sentence carries operational information (offset alignment, re-scan cost, link auth) that an agent needs; nothing is padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description bears full responsibility for return-value semantics, and it delivers: it names issueDetails sub-fields (allIssues, rules, pages, failedRules, pagination), explains severity ordering and per-element howToFix/criteria, and defines the pagination termination condition. An agent has everything needed to call and consume this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema coverage the baseline is 3, but the description adds real meaning on top: it clarifies the urls/testId exclusivity, the testId+offset pairing for paging, and that filters must stay constant across pages. It leaves some params (severity, pageUrl, include_rules) to the schema, which documents them adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ("Runs an accessibility scan on the given URLs") and immediately distinguishes its two operating modes: starting a scan via urls versus re-reading via testId. An agent understands both what it does and the dual nature of the tool without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the exactly-one-of constraint between urls and testId, explains when to use each, and proactively warns that re-passing urls to page starts a new scan, costs another run, and yields misaligned offsets. This is precisely the conditional guidance that prevents incorrect invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAnalyticsDataGet Analytics DataARead-onlyInspect
Summarises web automation test results in a date range: total tests, passed, failed, completed without a pass/fail mark, pass rate (passed over all tests, as the Insights page shows it), pass rate over tests marked passed or failed, average duration and distinct builds, grouped by status, browser, os, build, device or day. Computed from the automation test list (up to 5000 tests per call; the total is always exact).
| Name | Required | Description | Default |
|---|---|---|---|
| toDate | No | End date, YYYY-MM-DD. Defaults to today. | |
| groupBy | No | Dimension to group by. Defaults to status. | |
| fromDate | No | Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, non-destructive, closed-world), and the description usefully adds operational context: it is computed from the automation test list, capped at 5000 tests per call, yet the total is always exact, and pass rate is defined to match the Insights page. This clarifies how the numbers are derived rather than just repeating annotation hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence that opens with the verb and then lists metrics and grouping options compactly. Dense but every clause carries information; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description carries the burden of explaining returns, and it does so thoroughly by naming each aggregated metric and the possible grouping dimensions. Minor gap: it does not describe the shape of the returned records per group, but otherwise complete for a read-only analytics tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the parameter descriptions already document formats, defaults, and the 59-day retention limit for fromDate. The description adds no parameter-level detail beyond restating the date-range grouping, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Summarises web automation test results') and enumerates the exact metrics produced (total, passed, failed, pass rates, avg duration, distinct builds) plus the group-by dimensions. This makes it clearly distinguishable from siblings such as getTestTrends, getFlakyTests, and getAnalyticsErrors without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The date-range scope and aggregate nature imply when the tool applies, but there is no explicit when-to-use versus alternatives guidance and no mention of getTestTrends or getAnalyticsErrors as related options. Usage is left to inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAnalyticsErrorsGet Analytics ErrorsARead-onlyInspect
Summarises failed and errored web automation tests in a date range: counts by failure category and by status, and the failing tests themselves with build and environment. Failure categories exist only for about the last 7 days of tests; older failures are counted as uncategorised.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Maximum failing tests to list, 1 to 200. Defaults to 20. | |
| toDate | No | End date, YYYY-MM-DD. Defaults to today. | |
| fromDate | No | Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, non-destructive, non-open-world), so the description's job is to add context — and it does: the ~7-day window for failure categories and the fact older failures fall back to 'uncategorised' is a real, non-obvious data caveat. It stops short of noting ordering, latency, or empty-result behaviour.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler. The core purpose and return shape are front-loaded, and the second sentence is a genuine caveat that earns its place rather than padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the burden of describing returns and does so — counts by failure category and status, plus the failing tests with build and environment. Combined with the fully documented input schema and safety annotations, an agent has everything needed to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with limit, fromDate and toDate all fully documented including defaults and the 59-day bound. The description only restates 'date range' generically and adds no format or boundary detail beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (summarises) and a tightly scoped resource (failed and errored web automation tests) plus the exact breakdowns returned, which separates it from broad analytics tools. However, it never names the neighbouring tools (getAnalyticsData, getFlakyTests, getTestTrends), so an agent must infer the distinction from scope alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied rather than stated: the 'date range' framing and the failure-only scope let an agent infer it is for troubleshooting recent failures. There is no explicit when-to-use, no when-not-to-use, and no pointer to alternatives such as getTestTrends for pass/fail trends or getFlakyTests for intermittency.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAutomationTestBrowserConsoleLogsGet Automation Test Browser Console LogsARead-onlyInspect
Fetches Browser Console logs for a specific automation test. Note: Not available for mobile app automation tests.
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | The ID of the automation test | |
| platform | Yes | The platform type: "web" for browser automation, "app" for app automation (not supported) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered without the description. The description adds the mobile-platform limitation, which is real context, but says nothing about pagination, log retention windows, truncation, or what the returned logs look like. Useful but thin beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler, with the core purpose front-loaded and the platform caveat immediately after. Nothing could be removed without losing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter read-only tool this is close to adequate, but there is no output schema, so the description is the only place the return shape could be described (log entries, ordering, volume). It also omits any sibling routing for the four near-identical log tools, leaving the agent to guess.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both testId and platform are already documented in the schema, including the enum note that 'app' is unsupported. The description restates the mobile restriction but adds no new syntax or format guidance, so the baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Fetches Browser Console logs for a specific automation test.' That distinguishes it by log type from siblings like getAutomationTestCommandLogs and getAutomationTestNetworkLogs, though those alternatives are never named. It is clear on its own but leaves the agent to infer the sibling boundaries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives one useful exclusion ('Not available for mobile app automation tests'), which tells the agent when this tool does not apply. However, it never states when to reach for this tool versus the command/network/Selenium log siblings, so the positive selection condition is only implied.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAutomationTestCommandLogsGet Automation Test Command LogsCRead-onlyInspect
Fetches command logs for a specific automation test
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | The ID of the automation test | |
| platform | Yes | The platform type: "web" for browser automation, "app" for app automation | |
| sessionId | No | Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID) | |
| pageNumber | No | Page number to fetch (default: 1) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so safety is covered. The description contributes nothing beyond that: no mention of pagination behavior, log volume/ordering, or output shape. It neither contradicts nor enriches the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with no waste and the key resource front-loaded. It is efficient, though arguably too terse for a tool sitting in a crowded log-retrieval family.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and four parameters, the description should carry the load on what command logs actually contain, pagination expectations, and how this differs from sibling log tools. None of that is present, leaving the definition under-specified for a 4-param, paged retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, including a notably useful note that sessionId is required for the app platform and comes from listMobileSessions/getAutomationTestDetails. The description adds no parameter meaning of its own, so the schema-driven baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Fetches) and resource (command logs for a specific automation test), so the basic purpose is clear. However, it does not differentiate itself from near-identical siblings such as getSessionV2CommandLogs, getAutomationTestSeleniumLogs, getAutomationTestNetworkLogs, and getAutomationTestBrowserConsoleLogs, which an agent must choose between.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives no when-to-use guidance, no exclusions, and no routing to alternatives. In a cluster of four-plus log-retrieval tools this is a real hazard, since nothing tells the agent why it would pick command logs over console/network/Selenium logs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAutomationTestDetailsGet Automation Test DetailsBRead-onlyInspect
Fetches detailed information about an automation test by ID
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | The ID of the automation test | |
| platform | Yes | Optional: set to "app" to fetch details from the mobile API |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structured data. The description adds no further behavior beyond 'fetches details' — no note on which fields come back, whether results are cached, or the platform-routing behavior. With annotations doing the heavy lifting, this is merely adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with the verb and resource front-loaded and zero filler. Nothing to trim.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description must at least hint at what 'details' contains — it does not. Given a crowded sibling namespace of test-fetching tools, the definition also fails to position this tool relative to them, leaving the agent under-informed for a two-required-parameter call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the schema itself explains both testId and the platform enum's mobile-API effect. The description contributes only the vague phrase 'by ID', adding no meaning beyond the schema, so the baseline of 3 for high coverage applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Fetches detailed information about an automation test by ID'), which is clear enough to act on. However, it does not distinguish itself from the many nearby siblings that also fetch automation-test data (getAutomationTestCommandLogs, getAutomationTestNetworkLogs, getAutomationTestSeleniumLogs, getAutomationTestBrowserConsoleLogs) — those are log-specific, but the description never frames this tool as the general test-metadata lookup versus those.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance at all: nothing says whether this should be called before/after the log-fetching siblings or what prompts it. The only routing hint ('set to "app" to fetch details from the mobile API') is buried in the schema, not the description. An agent has to infer the whole call context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAutomationTestNetworkLogsGet Automation Test Network LogsBRead-onlyInspect
Fetches network logs for a specific automation test
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | The ID of the automation test | |
| platform | Yes | The platform type: "web" for browser automation, "app" for app automation | |
| sessionId | No | Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structured data. The description adds nothing beyond the annotations — no return format, log volume, or pagination behavior — so a baseline 3 is appropriate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler or redundancy. It is efficient, though its brevity comes at the cost of any usage or behavioral detail rather than being a model of informative concision.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read-only log fetch with a fully documented 3-parameter schema and no output schema, the annotations and schema carry most of the burden. The description still omits any usage context or output expectations, leaving it only minimally adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema itself documents testId, the platform enum, and the conditional sessionId requirement in detail. The description adds no parameter meaning beyond what the schema already provides, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb (fetches) and resource (network logs) scoped to a single automation test, which distinguishes it from the log siblings (browser console, command, Selenium logs) by resource type. It does not explicitly say how it differs from those siblings, so it falls short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus the other automation-test log tools, and no mention of prerequisites such as the app-platform sessionId requirement. The description leaves the agent to infer the selection criteria entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getAutomationTestSeleniumLogsGet Automation Test Selenium LogsARead-onlyInspect
Fetches Selenium logs (web), Appium logs (mobile/Appium), or instrumentation logs (mobile/XCUI/Espresso) for a specific automation test. Automatically detects XCUI and Espresso tests and fetches the correct log type.
| Name | Required | Description | Default |
|---|---|---|---|
| testId | Yes | The ID of the automation test to fetch logs for | |
| platform | Yes | The platform type: "web" for Selenium logs, "app" for Appium/instrumentation logs | |
| sessionId | No | Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the safe read-only profile (readOnlyHint=true, destructiveHint=false). The description adds useful behavior — the three log categories and the fact that XCUI/Espresso are auto-detected — but says nothing about log size, truncation, or availability windows for a log-retrieval tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with no filler, and the log-type mapping is front-loaded. Efficient, though the second sentence could be folded in without loss.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries the return-value burden and does identify what comes back (web/Appium/instrumentation logs). It is adequate for a three-parameter read tool, missing only edge details like empty-log or unavailable-session behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, including the important sessionId constraint (required for app platform, not the test ID) and the platform enum. The description adds no parameter guidance beyond the schema, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb ("Fetches") plus a specific resource (Selenium/Appium/instrumentation logs) scoped to "a specific automation test". However, it does not distinguish itself from close siblings like getAutomationTestBrowserConsoleLogs, getAutomationTestCommandLogs, or getAutomationTestNetworkLogs, so the agent must infer which log family each tool returns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied (use it to get logs for one test), and the note about auto-detecting XCUI/Espresso gives some routing help. But there is no explicit when-to-use/when-not or named alternative versus the other getAutomationTest*Logs siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getFlakyTestsGet Flaky TestsARead-onlyInspect
Lists tests that the insights engine marks as flaky (passing and failing intermittently) in a date range, with each test's flake rate, build and environment. Returns only flaky tests, plus how many tests were scanned. Flakiness is only computed for about the last 7 days of tests; the response says how many tests in the range had it.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Maximum flaky tests to return, 1 to 200. Defaults to 50. | |
| toDate | No | End date, YYYY-MM-DD. Defaults to today. | |
| fromDate | No | Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish the safe, read-only, closed-world profile. The description adds genuinely useful non-annotation context: flakiness is only computed for roughly the last 7 days, only flaky tests are returned, and the payload includes a scanned-test count with how many had flakiness computed. No auth or rate-limit detail, but the disclosure goes meaningfully beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with no filler; the what-it-returns content is front-loaded and the 7-day limitation is attached directly to the flakiness concept rather than buried at the end.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description carries the return-value burden and does so well, naming flake rate, build, environment, and the scanned count. It stops short of describing ordering or empty-result behavior, but for a zero-required-parameter read tool this is nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description still adds value by explaining a constraint the schema hints at only partially: the 7-day flakiness computation window versus the 59-day serviceable range, which tells the agent results will be sparse or absent for older fromDate values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Lists) and a precise resource (tests marked flaky, i.e. passing/failing intermittently) scoped to a date range, and enumerates the returned fields (flake rate, build, environment). An agent can distinguish this from report/exception siblings like getTestTrends or postTestExceptions without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The date-range scope and the 'returns only flaky tests' framing imply when to reach for this tool, and the 7-day flakiness window is a useful caveat. However, no alternative tool is named and there is no explicit when-not guidance, so usage is only implied rather than directed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteJobArtifactsGet HyperExecute Job ArtifactsBRead-onlyInspect
Retrieve all artifacts or categorized errors for a HyperExecute job or task.
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The HyperExecute job ID | |
| taskId | No | Optional: The HyperExecute task ID for categorized errors |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds only the two-mode behavior (all artifacts vs categorized errors); it says nothing about result volume, pagination, or how large artifact sets are returned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with zero filler. It is efficient, though bordering on under-specified for a tool with multiple retrieval modes and several sibling artifact tools.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read tool with annotations and no output schema, the description is minimally adequate. It does not define what 'categorized errors' contains or how this differs from downloadJobArtifact and getJobArtifactDetail, which is the main open question an agent would have.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already states that taskId is optional and drives categorized errors. The description restates this rather than adding format, constraints, or interaction details, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb (Retrieve) plus resource (artifacts / categorized errors) scoped to a HyperExecute job or task. However, it does not distinguish itself from close siblings like downloadJobArtifact or getJobArtifactDetail, so an agent cannot tell which artifact-retrieval tool to pick from the description alone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied: passing taskId selects the 'categorized errors' mode while jobId alone returns all artifacts. There is no explicit when-to-use, no mention of alternatives such as downloadJobArtifact/getJobArtifactDetail, and no note on limits or size of results.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteJobInfoShow HyperExecute JobBRead-onlyInspect
Fetches and displays information about a HyperExecute job including status, test results, and progress.
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The ID of the HyperExecute job |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds genuine value by enumerating returned content (status, test results, progress), which matters because there is no output schema, but it omits auth requirements, result freshness, or behavior for an unknown/invalid jobId.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with the verb and the return content, with no filler. It is slightly generic in the 'Fetches and displays' opening but wastes nothing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter read tool with annotations covering safety and no output schema, the description adequately conveys what the caller gets back. It falls short on sibling differentiation against the several other HyperExecute read tools and on any behavioral edge cases, leaving it minimum-viable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is a single parameter with 100% schema description coverage ('The ID of the HyperExecute job'), so the schema already carries the semantics and 3 is the baseline. The description only implies that jobId selects the job and adds no format or sourcing detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Fetches and displays information about a HyperExecute job') and even names the content returned (status, test results, progress), which distinguishes it from artifact/log/session fetchers in the sibling list. It does not, however, differentiate itself from analyzeHyperExecuteJob or listHyperExecuteJobs, so an agent must still infer which HyperExecute read tool to pick.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no prerequisites, and no mention of alternatives such as analyzeHyperExecuteJob or getHyperExecuteJobSessions. The agent gets a purpose but no routing signal.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteJobLiveStreamGet HyperExecute Job Live StreamBRead-onlyInspect
Stream live output from a running HyperExecute job. (UNDOCUMENTED - may change without notice)
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The running job ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/destructiveHint=false, so the safety profile is covered. The description adds useful behavioral signals: output is live/streaming rather than a one-shot fetch, and the tool is explicitly undocumented and unstable. It does not describe stream termination, blocking behavior, or error handling, so it is only partially transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core action and immediately followed by the stability caveat. Nothing is wasted and no filler is present.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a streaming tool this is somewhat thin: it does not explain how the stream is consumed or terminated, and there is no output schema to compensate. Annotations cover the safety profile and the single parameter is fully documented, so it is minimally adequate but leaves the streaming semantics under-specified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the single jobId parameter is documented as 'The running job ID' in the schema. The description adds no syntax, format, or constraint detail beyond what the schema already provides, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Stream live output') scoped to a 'running HyperExecute job'. The words 'live' and 'running' naturally distinguish it from siblings like getHyperExecuteJobInfo or getHyperExecuteStageLogs, though it stops short of explicitly naming those alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use / when-not-to-use guidance, and no alternative is named. The only caveat, 'may change without notice,' is a stability warning rather than usage direction, so the agent must infer when a live stream is preferable to the batch log tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteJobSessionsShow HyperExecute Job SessionsCRead-onlyInspect
Fetches and displays session information for a HyperExecute job.
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The ID of the HyperExecute job | |
| limit | No | Maximum number of sessions to return (default: 10) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds nothing beyond that — no mention of pagination via 'limit', result ordering, or whether sessions include live versus completed runs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler or redundancy. It is efficient, though the brevity reflects under-specification rather than disciplined conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter read tool with full annotation coverage and no output schema, the description is minimally adequate. It omits what a 'session' represents in this context and how results are scoped or ordered, which would help disambiguate it from the many adjacent HyperExecute readers.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both jobId and the limit default of 10 are already documented in the schema. The description contributes no additional parameter meaning, which is the expected baseline when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a clear verb ('Fetches and displays') plus a specific resource ('session information for a HyperExecute job'). The resource distinguishes it from job-level siblings like getHyperExecuteJobInfo and listHyperExecuteJobs, but the description does not explicitly contrast with them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus sibling options such as getHyperExecuteJobInfo, getHyperExecuteStage, or getHyperExecuteJobLiveStream, and no stated preconditions or exclusions. The agent must infer the context from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteStageGet HyperExecute StageARead-onlyInspect
Retrieve the stages of a HyperExecute task (cache/prerun/scenario/etc.). Pass a stageId to return just that one stage. Stages are keyed by their parent task, so taskId is required.
| Name | Required | Description | Default |
|---|---|---|---|
| taskId | Yes | The HyperExecute task ID whose stages to fetch | |
| stageId | No | Optional: return only the stage with this ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is carried by structured data. The description adds the structural insight that stages are keyed by parent task, but says nothing about return shape, pagination, or error behavior when a stageId is not found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, all earning their place, with the core purpose front-loaded before the parameter guidance. No redundancy or filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, two-parameter tool with full schema coverage and annotations covering safety, the description gives enough context to call it correctly. The only minor gap is that return content is undescribed, which is tolerable given there is no output schema and the resource is intuitive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning beyond the schema by explaining WHY taskId is required (stages are keyed by their parent task) and by indicating the optional stageId narrows the result to a single stage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Retrieve) and resource (stages of a HyperExecute task) and clarifies the resource with examples (cache/prerun/scenario). An agent can distinguish it from getHyperExecuteTask and getHyperExecuteStageTasks, though it never names those siblings explicitly to reinforce the distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear conditional guidance for the optional stageId ('Pass a stageId to return just that one stage') and explains when taskId is required ('Stages are keyed by their parent task'). It does not name alternative tools or a when-not-to-use case, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteStageLogsGet HyperExecute Stage LogsBRead-onlyInspect
Retrieve logs for a HyperExecute stage. (UNDOCUMENTED - may change without notice)
| Name | Required | Description | Default |
|---|---|---|---|
| stageId | Yes | The stage ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already establish this is a safe, read-only, non-destructive, closed-world operation, so the bar is lower. The description adds one genuinely new behavioral fact – the endpoint is undocumented and may change without notice – which is useful for an agent assessing reliability, but it omits anything about log scope, size, or pagination.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, purpose front-loaded first, with a caveat second. Nothing is padded or redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and only one parameter, the definition is minimally serviceable, but it never indicates what the returned logs contain or how they are scoped (per stage, per run). The stability warning is a nice touch but does not compensate for the absent return expectations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single stageId parameter, so the schema already carries the parameter documentation; there is only a minimal 'The stage ID' description. The tool description adds no format or sourcing detail beyond the schema, making the baseline 3 appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Retrieve) and resource (logs for a HyperExecute stage), so the purpose is unambiguous. However, it does not distinguish itself from the closely related sibling getHyperExecuteTaskLogs or clarify its relationship to getHyperExecuteStage/getHyperExecuteStageTasks, leaving an agent to guess which log-retrieval tool applies at which granularity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus the many sibling log and stage tools such as getHyperExecuteTaskLogs or getHyperExecuteJobArtifacts. The only added sentence concerns API stability, not selection criteria or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteStageTasksGet HyperExecute Stage TasksBRead-onlyInspect
Retrieve tasks for a HyperExecute stage. (UNDOCUMENTED - may change without notice)
| Name | Required | Description | Default |
|---|---|---|---|
| stageId | Yes | The stage ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds the notable 'UNDOCUMENTED - may change without notice' stability caveat, which is genuine behavioral information beyond annotations, but says nothing about return contents, pagination, or auth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single short sentence with the stability caveat appended. Front-loaded and zero waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple single-parameter read tool with annotations covering safety and a fully described schema, the core need is met. However, no output schema exists and the description doesn't say what the tasks response contains, and it omits routing guidance among several HyperExecute task/stage siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the single stageId parameter fully documented in the schema. The description adds no syntax or format detail, so baseline 3 applies when the schema carries the load.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Retrieve) and resource (tasks for a HyperExecute stage), which clearly identifies the operation. It doesn't explicitly differentiate from close siblings like getHyperExecuteTask (singular) or getHyperExecuteStage, but the resource scope is clear enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this versus siblings such as getHyperExecuteTask or getHyperExecuteStage. The '(UNDOCUMENTED - may change without notice)' caveat is a warning, not usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteTaskGet HyperExecute TaskARead-onlyInspect
Retrieve details for a HyperExecute task by task ID. (UNDOCUMENTED - may change without notice)
| Name | Required | Description | Default |
|---|---|---|---|
| taskId | Yes | The task ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this is a safe read (readOnlyHint=true, destructiveHint=false, openWorldHint=false). The description adds a genuinely useful behavioral signal not present in the structured data: the tool is UNDOCUMENTED and may change without notice, warning the agent about stability risk.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence states the action plus a compact parenthetical warning. Nothing is padded and the purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read tool with full schema coverage and annotations carrying the safety profile, the description is adequate. It stops short of describing the shape of returned details or how it differs from the several similar HyperExecute retrieval tools, leaving some ambiguity in a dense sibling set.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with a single required taskId parameter documented in the schema. The description restates the task ID lookup without adding format, validation, or sourcing details, so it earns the baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Retrieve) and resource (HyperExecute task details) with the lookup key (task ID). It is reasonably distinguishable from siblings like getHyperExecuteJobInfo or getHyperExecuteStageTasks, but it never explicitly contrasts itself with those tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus the many sibling getHyperExecute* tools, and no prerequisites or context is given. The agent must infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getHyperExecuteTaskLogsGet HyperExecute Task LogsBRead-onlyInspect
Retrieve logs for a HyperExecute task. (UNDOCUMENTED - may change without notice)
| Name | Required | Description | Default |
|---|---|---|---|
| taskId | Yes | The task ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description does add one genuinely useful behavioral trait beyond the annotations: the '(UNDOCUMENTED - may change without notice)' caveat warns that output/behavior is unstable. It still says nothing about log availability timing, size, or format, so it is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short clauses with zero padding, and the core purpose is front-loaded ahead of the caveat. Nothing is repeated from the title or schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-param read tool with full schema coverage and annotations, the description covers the essentials, and no output schema exists so return values need not be explained. It is still missing the sibling disambiguation and any note on when logs become available, which leaves a real gap in a crowded log-retrieval family.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is a single parameter with 100% schema description coverage, so the schema already documents 'taskId'. The description adds no syntax, format, or sourcing detail (e.g. where to obtain a task ID) beyond what the schema provides, matching the baseline 3 for schema-covered params.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Retrieve logs for a HyperExecute task'), so the agent knows the operation. However, it does not distinguish itself from close siblings such as getHyperExecuteStageLogs or getHyperExecuteTask, and does not clarify whether logs are task-scoped or stage-scoped, which is the key ambiguity in this family.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance, no mention of prerequisites (e.g. whether the task must be completed or still running), and no routing to alternatives like getHyperExecuteStageLogs when stage-level logs are wanted. The agent must infer usage entirely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getJobArtifactDetailGet Job Artifact DetailBRead-onlyInspect
Get details for a specific artifact within a HyperExecute job.
| Name | Required | Description | Default |
|---|---|---|---|
| jobId | Yes | The HyperExecute job ID | |
| artifactId | Yes | The artifact ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds no additional behavioral context such as authentication needs, pagination, or what happens for missing artifacts, but it also does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single clear sentence, front-loaded with the action and resource. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter read operation with complete schema and read-only annotations, the description is minimally sufficient. It does not explain what artifact details are returned, but no output schema exists to carry that burden either.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both jobId and artifactId are fully documented in the schema. The description adds no further meaning to these parameters, which is acceptable given the high schema coverage baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: get details for a specific artifact within a HyperExecute job. It distinguishes itself from list-oriented siblings like getHyperExecuteJobArtifacts by emphasizing a specific artifact, though it does not explicitly name alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides no guidance on when to use this tool versus alternatives such as getHyperExecuteJobArtifacts or downloadJobArtifact. The intended context is implied only by the phrase 'specific artifact'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getLiveSessionLogsGet Live Session LogsARead-onlyInspect
Retrieve the request and response log of a test session, while it runs or after it ends. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The web automation test ID (e.g. DA-WIN-...) or session ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds real behavioral context beyond annotations: that app automation sessions return a not-found error and which sibling carries app-session media. Annotations already declare readOnly/openWorld=false, so the description's job is smaller, but it could still note log retention, size limits, or completeness of the live log.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly packed sentences with the core capability front-loaded, immediately followed by the platform restriction and the alternative. No filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, no output schema, and annotations covering the safety profile, the description is nearly complete for correct invocation. The remaining gap is output shape (log format, truncation), which is minor for a read-only retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the single sessionId already documents the accepted ID format (DA-WIN-... or session ID). The description adds no format or identifier nuance beyond that, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource ('Retrieve the request and response log of a test session') with explicit scope ('while it runs or after it ends'). It explicitly distinguishes itself from mobile siblings by restricting to web automation and pointing app-session consumers to listMobileSessions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
States the exact when: web automation (Selenium, Playwright, Cypress) only, and the when-not: app automation returns not found, with the alternative listMobileSessions called out for video/screenshots. This is explicit routing, not inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getMobileCapabilityGeneratorGet Mobile Capability GeneratorCRead-onlyInspect
Retrieve capability generation data for mobile automation test configuration.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds nothing beyond that — no note on whether the capability payload reflects live org state, whether it can fail on unconfigured accounts, or what a typical response looks like.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler or repetition. It is appropriately sized, though its brevity comes at the cost of substance rather than from efficient editing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the burden of explaining what 'capability generation data' consists of, and it does not. For a zero-parameter getter the agent cannot infer the return shape or any gating condition from anywhere else in the definition.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate; baseline 4 applies. No parameter-level confusion is possible.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
It states a verb (Retrieve) and a nominal resource (capability generation data for mobile automation test configuration), so the domain is identifiable. However, 'capability generation data' is undefined jargon that restates the tool name rather than explaining what is actually returned, leaving the agent unable to distinguish it confidently from siblings like getMobileConcurrency or getSupportedPlatforms.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description contains no when-to-use guidance, no prerequisites, and no mention of any alternative tool. Given the large sibling set of mobile/test-configuration getters, the absence of routing guidance is a real gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getMobileConcurrencyGet Mobile ConcurrencyARead-onlyInspect
Retrieve the current concurrency usage and limits for mobile automation (real device testing).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=false, so the safe-read profile is covered. The description adds the useful scope that the numbers are mobile/real-device specific and that it returns both usage and limits, but says nothing about whether values are a live snapshot or cached.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence with no filler; the resource and its scope appear immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description does name what comes back (usage and limits), which covers the core need for a parameterless read tool. It stops short of describing freshness, units, or per-platform breakdown.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Zero parameters, so the schema carries no semantics to supplement and the baseline is 4. Nothing in the description misrepresents the empty input contract.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Retrieve) and resource (current concurrency usage and limits) scoped to mobile automation / real device testing. It is clear, but it never explicitly distinguishes itself from the sibling getOrgConcurrency, leaving the mobile-vs-org boundary to inference.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The scoping phrase 'for mobile automation (real device testing)' implies when the tool is relevant, but there is no explicit when-to-use, prerequisite, or named alternative such as getOrgConcurrency. Usage is inferable rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getOrgConcurrencyGet Org ConcurrencyBRead-onlyInspect
Retrieve the current concurrency usage and limits for the organization on web automation (Selenium/Cypress).
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the agent knows this is a safe read operation. The description adds the context that the concurrency is for web automation, which is useful. It does not, however, disclose behavioral traits beyond what annotations provide, such as whether this is a real-time lookup or if the response includes historical data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and scope with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity (no params, no output schema) and the annotations covering safety, the description is adequate. However, it lacks a crucial piece: an explicit statement that this tool is for web automation (Selenium/Cypress) and not mobile, which would help the agent choose between it and getMobileConcurrency.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the baseline is 4. The description correctly implies no parameters are needed, and it clarifies the scope (organization-level, web automation) without introducing unnecessary parameter details.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb 'retrieve' and resource 'current concurrency usage and limits for the organization on web automation (Selenium/Cypress)', clearly scoping it to web automation. However, it does not explicitly differentiate itself from the sibling 'getMobileConcurrency', which retrieves mobile concurrency limits.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit guidance on when to use this tool versus alternatives. While the tool name and description imply it should be used to get org concurrency, the presence of 'getMobileConcurrency' as a sibling creates ambiguity that is not resolved – the description does not state that this tool is for web (Selenium/Cypress) as opposed to mobile.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getSessionScreenshotsGet Session ScreenshotsARead-onlyInspect
Retrieve command-by-command screenshots captured during a test session. Returns screenshot URLs for each step. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The web automation test ID (e.g. DA-WIN-...) or session ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnlyHint true, destructiveHint false, openWorldHint false). The description adds meaningful behavioral context beyond that: it specifies the web-only scope, the error behavior for app sessions, and that it returns per-step screenshot URLs. It does not cover pagination or auth requirements, but for a read-only tool this is a strong addition.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with zero waste: the core action and return value are front-loaded, followed by scope constraints and the alternative tool. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description still tells the agent what to expect (screenshot URLs for each step). It also fully covers the key usage boundary (web vs app automation) and points to the sibling that handles app sessions, so an agent has all needed context to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single sessionId parameter, which already documents its expected format (e.g., DA-WIN-...) and dual meaning (test ID or session ID). The description adds no further parameter detail, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (retrieve) and resource (command-by-command screenshots captured during a test session), and immediately clarifies the output (screenshot URLs for each step). It also distinguishes itself from sibling getSessionVideo by focusing on screenshots and from listMobileSessions by confining scope to web automation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use this tool (web automation sessions only) and when not to use it (app automation sessions return not found). It names the alternative tool for app sessions (listMobileSessions), leaving no ambiguity about routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getSessionV2CommandLogsGet Session V2 Command LogsARead-onlyInspect
Retrieve v2 command execution logs for a test session. Returns structured JSON with enhanced detail compared to v1 logs. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The web automation test ID (e.g. DA-WIN-...) or session ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is low. The description adds genuinely useful behavior beyond them: the web-only constraint, the concrete failure mode (not found) for app sessions, and the return shape ('structured JSON'). It does not mention volume, pagination, or retention limits for logs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, zero filler, and the core purpose is front-loaded before the scope constraint and the redirect to the alternative. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter log retrieval tool with no output schema, the description covers purpose, scope, and a rough return shape, which is close to sufficient. Minor gaps remain around log volume/pagination expectations, but nothing blocks correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter, and schema coverage is 100% – the schema already documents sessionId as a web automation test ID or session ID. The description adds no format nuance, examples, or validation rules beyond that, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource (retrieve v2 command execution logs for a test session) and explicitly positions itself against the v1 variant by promising 'enhanced detail compared to v1 logs'. It also carves out a hard scope boundary (web automation only) that separates it from log-retrieval siblings like getAutomationTestCommandLogs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives an explicit when-not ('an app automation session returns not found') and routes the agent to the alternative for that case ('listMobileSessions carries the video and screenshot links of app sessions'). It does not, however, clarify when to prefer this over getAutomationTestCommandLogs or the other log endpoints for web sessions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getSessionV2SeleniumLogsGet Session V2 Selenium LogsARead-onlyInspect
Retrieve v2 Selenium/WebDriver grid request and response logs for a test session. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The web automation test ID (e.g. DA-WIN-...) or session ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnly, non-destructive, closed-world). The description adds a genuinely useful behavioral fact beyond that: app automation sessions produce a not-found result rather than logs, which prevents a wasted call. Return format and pagination are still unstated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences: the core action first, then the web-only constraint and the escape hatch to listMobileSessions. Every clause earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, so the description ideally would hint at log shape or volume, which it does not. However, it fully covers the tool's scope, the platform constraint, and the failure case, which is sufficient for a single-parameter log retrieval call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and there is a single required parameter, so the schema carries the semantic load (session ID, with the DA-WIN- example). The description only confirms 'for a test session' and adds no format or syntax detail beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Retrieve) and resource (v2 Selenium/WebDriver grid request and response logs) scoped to a test session. It clearly separates web automation from app automation sessions, though it does not explicitly disambiguate itself from the similarly named sibling getSessionV2CommandLogs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the when-not: app automation sessions return not found. It also names an alternative (listMobileSessions) for video and screenshot links, though it does not route the agent among the other v2/automation log siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getSessionVideoGet Session VideoARead-onlyInspect
Retrieve the video recording URL for a completed test session. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The web automation test ID (e.g. DA-WIN-...) or session ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered. The description adds genuine behavioral context beyond that: it only succeeds for completed web sessions and fails with 'not found' for app sessions. It does not cover auth requirements or whether the URL is time-limited, so it lands just short of a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler, front-loaded with the core action followed by the platform constraint and fallback. Every clause earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter read-only lookup with no output schema, the description states what is returned (the video recording URL), the precondition (completed session, web automation), and the failure mode. Nothing an agent needs to invoke it correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter and schema description coverage is 100%, so the schema already documents sessionId including its format examples. The description adds no syntax or format detail beyond that. Baseline 3 is correct.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource ('Retrieve the video recording URL for a completed test session') that an agent can immediately distinguish from sibling tools like getSessionScreenshots or getSessionV2SeleniumLogs. Scope ('completed test session') is stated up front.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly restricts usage to web automation sessions and states the negative case (app automation returns not found), then routes the agent to the correct alternative (listMobileSessions) for app sessions. This is exactly the when/when-not/alternative pattern.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getSuiteOverviewGet Suite OverviewBRead-onlyInspect
Gets comprehensive suite overview for a project including run counts, latest status, scores, schedule information, and active calls.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | The project ID to get suite overview for |
Output Schema
| Name | Required | Description |
|---|---|---|
| suites | No | |
| project_id | No | |
| scheduled_runs | No |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is lower. The description adds what data categories are aggregated, which is useful, but does not disclose freshness, aggregation scope, or authorization requirements beyond the annotations' coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single efficient sentence that front-loads the verb and resource and packs the returned content into a compact list. No wasted clauses, though the enumerated payload detail is arguably redundant with the output schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, return-value details are unnecessary, and the single required parameter is fully documented in the schema. The definition is complete enough to invoke correctly; only the absence of routing context against sibling read tools keeps it from a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter (projectId) and schema description coverage is 100%, so the schema fully documents it. The description adds nothing about the parameter beyond what the schema already states, which is the expected baseline at full coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb ('Gets') plus specific resource ('suite overview for a project') with an enumerated list of what the overview contains (run counts, latest status, scores, schedule info, active calls). It is distinguishable from mutation siblings like triggerSuiteRun, but does not differentiate itself from read-oriented siblings such as tm_get_coverage_summary or getTestTrends.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance is given: no statement of when a suite overview is preferable to getTestTrends, tm_get_coverage_summary, or the various HyperExecute job queries. The agent must infer the use case from the description alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getSupportedPlatformsGet Supported PlatformsARead-onlyInspect
Lists the OS, browser and device combinations LambdaTest supports, to find valid os, browser and version values for a test configuration. With no filters it returns a compact summary: each desktop OS with its browsers (version count, newest and oldest) and resolutions, and each mobile platform with its device count and OS versions. Set os (and optionally browser) for the full version list, or os=ios / os=android for device names.
| Name | Required | Description | Default |
|---|---|---|---|
| os | No | OS name to expand, matched case-insensitively as a substring, e.g. "Windows 11", "Sonoma", "android" | |
| type | No | Only desktop OSes or only mobile platforms | |
| browser | No | Browser to expand on desktop, e.g. "chrome", "firefox", "edge", "safari" |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds real behavioral context the annotations don't: the exact shape of the unfiltered response (per-OS browser counts, newest/oldest versions, resolutions; per-mobile device counts and OS versions) and how filters change the output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads what the tool returns, then the filter modes, with no filler sentences. The second and third sentences are dense (nested counts, resolutions, version ranges), but every clause carries information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by describing the return shape in each mode, and annotations cover the safety profile. Missing only minor operational details such as result size or any cap on the unfiltered summary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so per-parameter meaning is already documented and the baseline is 3. The description goes beyond it by explaining parameter interaction — that os+browser yields a full version list while os=ios/os=android yields device names — which the schema's individual field descriptions do not convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Lists the OS, browser and device combinations LambdaTest supports') plus the downstream purpose ('to find valid os, browser and version values for a test configuration'). This distinguishes it from nearby siblings like listDevicesForTesting and getMobileCapabilityGenerator, which serve different lookups.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly explains the three invocation modes: no filters for a compact summary, os (optionally browser) for full version lists, os=ios/os=android for device names. That is strong guidance on how to call it, but it never states when to prefer this tool over listDevicesForTesting or getMobileCapabilityGenerator.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
getTestTrendsGet Test TrendsARead-onlyInspect
Daily trend of web automation test results in a date range: for each day, total tests, passed, failed, completed without a pass/fail mark, pass rate (passed over all tests), pass rate over tests marked passed or failed, and average duration.
| Name | Required | Description | Default |
|---|---|---|---|
| toDate | No | End date, YYYY-MM-DD. Defaults to today. | |
| fromDate | No | Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. With no output schema present, the description carries real weight by disclosing the returned metric set and, importantly, clarifying that 'completed without a pass/fail mark' is a distinct bucket and that two different pass-rate definitions are returned — a genuinely ambiguous point for an agent interpreting results. It stops short of stating retention/aggregation behavior beyond what the schema already says.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence that names the resource and scope first, then enumerates outputs. The list is dense but every clause maps to a distinct returned value, so nothing is filler; it could be broken up for scannability but is not padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-optional-parameter read tool with no output schema, the input side is fully covered by the schema and the output side is covered by the description's metric enumeration. What is missing is orientation — whether this is suite-wide or scoped to a project/test set, and how it relates to the other analytics tools — which keeps it short of complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both optional parameters fully documented including formats, defaults, and the 59-day retention limit. The description adds nothing about fromDate/toDate, so this is the baseline 3 case where the schema does all the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Daily trend of web automation test results in a date range') and enumerates exactly what each day's bucket contains. It is unambiguous on its own, but it never contrasts itself with adjacent analytics tools such as getFlakyTests, getAnalyticsData, or getSuiteOverview, so sibling differentiation is left to the caller.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no when-to-use guidance at all: no mention of when trend data beats a point-in-time report, no exclusions, and no alternatives named among the ~70 siblings. The date-range expectation is only inferable from the parameter names, not stated as usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listDevicesForTestingList Devices for TestingARead-onlyInspect
Lists the real devices available on LambdaTest for mobile app testing, 50 per page by default. Filter by region, OS, device name or OS version, and page with limit and offset. The text gives the total that matched.
| Name | Required | Description | Default |
|---|---|---|---|
| os | No | Operating system filter | |
| limit | No | Devices per page, 1 to 200 (default 50) | |
| offset | No | Devices to skip (default 0) | |
| region | No | Device region (default: us) | |
| deviceName | No | Device name to match, case-insensitively as a substring, e.g. "Galaxy S24", "iPhone 15", "Pixel" | |
| platformVersion | No | OS version to match, e.g. "14" (matches 14 and 14.x) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds real value beyond that: the default page size of 50 and the fact that the response text reports the total number matched, which is the only source of return-shape information since there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences with zero filler: what it lists, the default page size, the filterable dimensions, and the returned total. The resource and default are front-loaded so the agent gets the essential facts immediately.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only paginated list tool with a fully documented schema and no output schema, the description covers pagination defaults, filter dimensions, and the returned total count. It is nearly complete, lacking only a note on ordering or how to interpret the matched-total value.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and every parameter is documented in the schema, including the substring/case-insensitive match semantics and the '14 matches 14.x' version behavior. The description only restates those same filters, so the baseline of 3 is appropriate with no added syntax or format detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Lists the real devices available on LambdaTest for mobile app testing', which clearly identifies a device-catalog read. It does not distinguish itself from the closest sibling (getSupportedPlatforms), which also concerns available platforms, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied rather than stated: the description explains that you can filter by region/OS/name/version and paginate, which hints at the discovery/search scenario. But it never says when to prefer this tool over getSupportedPlatforms or other listing siblings, and offers no exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listHyperExecuteJobsList HyperExecute JobsARead-onlyInspect
List the account's HyperExecute jobs, newest first, with offset pagination and an optional status filter. The status filter looks at the 200 most recent jobs.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Maximum number of jobs to return, 1 to 200 (default 10) | |
| offset | No | Number of jobs to skip, after the status filter (default 0) | |
| status | No | Only jobs with this status (e.g. completed, failed, running, skipped), among the 200 most recent |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint, destructiveHint=false, openWorldHint=false). The description adds behavior beyond that: results are newest-first, pagination is offset-based, and critically the status filter only ranges over the 200 most recent jobs, which would otherwise cause silently incomplete results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, with the core scope front-loaded and the filter limitation placed immediately after so it is read before the agent commits to a query.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still gives an agent enough to call the tool correctly: scope, ordering, pagination and the filter window. It stops short of indicating what a returned job record contains, but that is a minor omission for a straightforward list endpoint.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all three parameters are already documented in the schema, making 3 the baseline. The description's restatement of the 200-job filter window duplicates the status parameter description rather than adding new syntax or format detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List the account's HyperExecute jobs') plus the ordering and pagination model, which cleanly separates it from the many single-job siblings like getHyperExecuteJobInfo and getHyperExecuteJobSessions. It does not explicitly name an alternative for the job-detail use case, but the list scope is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied rather than stated: an agent can infer this is for browsing jobs rather than inspecting one, and the 'status filter looks at the 200 most recent jobs' caveat is a genuinely useful boundary. However, there is no explicit when-to-use vs when-not, nor a pointer to the detail/artifact siblings for drilling into a specific job.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listMobileBuildsList Mobile BuildsARead-onlyInspect
Lists mobile app automation builds, newest first, 10 per page by default. Page with limit and offset; the result gives the total the API reports. The API returns only builds from the last 90 days, so total can be higher than the builds you can page through.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Rows per page, 1 to 100 (default 10) | |
| offset | No | Rows to skip (default 0) | |
| status | No | Only builds with this status, e.g. completed, running, passed, failed |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structure. The description adds genuinely non-obvious behavior beyond that: newest-first ordering, a 10-per-page default, and most importantly that the API only returns builds from the last 90 days so the reported total can exceed what is pageable. That retention/consistency caveat is exactly the kind of disclosure annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, no filler, and the ordering/default-page-size facts are front-loaded before the caveat. Every sentence carries distinct information: what is listed, how to page, and the retention limitation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the burden of describing returns, and it does: ordering, page size, and the behavior of the total relative to the pageable set. Combined with full schema coverage on all three parameters and safe-read annotations, an agent has everything needed to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so limit and offset are already documented and the baseline would be 3. The description goes slightly beyond the schema by explaining the total-vs-pageable interaction that arises from the 90-day truncation, which affects how an agent should interpret paging results. The status parameter is not addressed at all in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource ('Lists mobile app automation builds') plus ordering ('newest first') and default page size, which is far more than a restated title. It does not, however, name or implicitly distinguish itself from nearby siblings such as listMobileSessions, listUploadedApps, or listHyperExecuteJobs, leaving the resource name alone to do the disambiguation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no explicit when-to-use / when-not-to-use statement or reference to an alternative tool for retrieving build data, so tool selection is left to inference from the resource name. The description does give practical usage mechanics (paging with limit/offset) and a conditional caveat about the 90-day window, which is useful but not selection guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listMobileSessionsList Mobile SessionsARead-onlyInspect
Lists mobile app automation sessions (tests), newest first, 10 per page by default, with their video, screenshot and log links. Page with limit and offset, or filter by buildId or status; the result gives the total the API reports.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Rows per page, 1 to 100 (default 10) | |
| offset | No | Rows to skip (default 0) | |
| status | No | Only sessions with this status, e.g. passed, failed, completed, error | |
| buildId | No | Only sessions of this build (build_id from listMobileBuilds) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is lower. The description still adds genuine behavioral context: newest-first ordering, a 10-per-page default, and that video/screenshot/log links are included in each row.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense, front-loaded sentences that lead with what is listed and then cover ordering, paging, filtering, and return contents. No filler and no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully tells the agent the response carries video/screenshot/log links and a total count, and it explains the pagination contract. It is essentially complete for a simple read-only list tool, though ordering ties and auth expectations are unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already documents all four parameters including defaults and the buildId provenance. The description only restates that limit/offset page and buildId/status filter, adding no syntax or format detail beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Lists) and resource (mobile app automation sessions/tests), and adds useful scope detail such as ordering and default page size. It does not explicitly name a near sibling (e.g., listMobileBuilds or getHyperExecuteJobSessions) to disambiguate, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains how to use the tool (page with limit/offset, or filter by buildId or status) but never states when to choose it over alternatives or any prerequisites. Usage is implied rather than directed, which is minimum-viable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listTunnelsList TunnelsARead-onlyInspect
Retrieve all active LambdaTest tunnel connections for the account. Useful for checking which tunnels are running before starting a new one or debugging local testing connectivity.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structured data. The description adds the 'active' scoping qualifier and intent context, but says nothing about result size, pagination, or what an account with no tunnels returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, zero filler, with the core action front-loaded before the use-case rationale. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a parameterless read-only listing tool whose annotations already cover the safety profile, this is nearly complete. The only mild gap is that with no output schema present, the description does not indicate the shape or fields of the returned tunnel entries.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to compensate for; the baseline for a 0-parameter tool is 4. The description correctly implies account-scoped retrieval with no input needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — 'Retrieve all active LambdaTest tunnel connections for the account' — which is unambiguous. It does not explicitly distinguish itself from related tunnel siblings such as stopTunnel or analyzeAppViaTunnel, so it falls just short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete usage contexts: checking running tunnels before starting a new one, and debugging local testing connectivity. However, it names no alternative tools and states no exclusions, so it is clear context without full when/when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
listUploadedAppsList Uploaded AppsARead-onlyInspect
Retrieve all apps uploaded to LambdaTest for mobile testing. Use type=android/ios for real devices, emulator/simulator for virtual devices.
| Name | Required | Description | Default |
|---|---|---|---|
| type | No | App type: android/ios for real devices, emulator/simulator for virtual | |
| level | No | Scope: user (default, apps you uploaded) or team (apps across your organization; sent to the API as level=org) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered and the bar is lower. The description adds the scope of what is listed (all uploaded mobile apps) but says nothing about result size, pagination, or whether all items are returned at once.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, with the core purpose front-loaded and the parameter hint following. Every clause carries information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only, two-optional-parameter list tool with no output schema and full annotation coverage, this is close to sufficient: purpose, platform scope and enum semantics are all present. Minor gaps remain around result size/pagination and how the returned apps relate to downstream tools like analyzeAppViaTunnel.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and both enums are fully documented in the schema, including the level=org mapping that the description omits entirely. The description's type sentence largely restates the schema's own wording, so it adds little beyond structured data — baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource (retrieve all uploaded mobile-testing apps) with the platform named, which distinguishes it from siblings like listMobileBuilds or listMobileSessions. It does not explicitly name or contrast with any sibling (e.g. upload_app, deleteUploadedApps), so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The second sentence gives real selection guidance for the type parameter (android/ios = real devices, emulator/simulator = virtual), which is genuine usage help. However there is no guidance on when to use this tool versus alternatives, and the level parameter is never mentioned, so usage remains implied rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
postSessionExceptionsPost Session ExceptionsAInspect
Upload exception/error logs from the client machine for a test session, by session ID. Use this to attach assertion failures or unexpected errors to a session. The result reads the test's exceptions back to confirm they were stored. The automation dashboard may not show exceptions uploaded this way after the test ended.
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | The exceptions to attach: {"exception": ["first message", "second message"]} | |
| sessionId | Yes | The session ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare this as a non-read-only, non-destructive, non-open-world write. The description adds genuinely useful behavior beyond that: the result reads exceptions back to confirm storage, and — critically — that the automation dashboard may not reflect exceptions uploaded after the test ended. That delayed-visibility caveat is real operational context the annotations cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four sentences, front-loaded with the core action and scope, and each subsequent sentence adds distinct value (use case, verification behavior, caveat). Minor slack, but no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description usefully explains the read-back confirmation behavior, and annotations cover the safety profile. For a nested-parameter mutation tool this is close to complete; permission requirements for uploading to an arbitrary session are the only noticeable omission.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the nested `data.exception` structure is fully documented with an example, so the schema carries the parameter semantics. The description only restates 'by session ID' and adds no format or constraint details beyond what the schema provides; baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: 'Upload exception/error logs from the client machine for a test session, by session ID.' The session scoping implicitly separates it from the sibling postTestExceptions, which handles test-level exceptions, though that sibling is never named explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear when-to-use guidance: 'Use this to attach assertion failures or unexpected errors to a session.' However, it offers no explicit when-not condition or named alternative (e.g. postTestExceptions for the test-level case), so the agent must infer the split.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
postTestExceptionsPost Test ExceptionsAInspect
Upload exception logs for a specific test (by test ID) rather than by session ID. The result reads the test's exceptions back to confirm they were stored. The automation dashboard may not show exceptions uploaded this way after the test ended.
| Name | Required | Description | Default |
|---|---|---|---|
| data | Yes | The exceptions to attach: {"exception": ["first message", "second message"]} | |
| testId | Yes | The test ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations cover the safety profile (readOnlyHint=false, destructiveHint=false, openWorldHint=false), so the description only needs to add behavior beyond that — and it does, disclosing that the call reads the exceptions back as a storage confirmation and that dashboard visibility may be unreliable post-test. It does not cover idempotency, overwrite semantics, or auth requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with the action and the distinguishing scope, followed by verification behavior and the one operational caveat. No sentence is redundant or padded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter mutation with annotations covering safety and a fully documented nested schema, the definition supplies the distinguishing scope, the read-back behavior, and the post-test visibility caveat. The read-back partially compensates for the absence of an output schema, though the exact response shape remains unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the nested data object, its required 'exception' array, and item constraints are fully documented in the schema. The description only restates that identification is by test ID, adding no syntax or format detail beyond the structured fields. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (upload exception logs) with an explicit scope qualifier ('for a specific test (by test ID) rather than by session ID'). This directly distinguishes it from the sibling postSessionExceptions without needing to open either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear selection rule against the session-based alternative and a caveat about when the result may be misleading ('automation dashboard may not show exceptions uploaded this way after the test ended'). It does not name the sibling tool explicitly or state prerequisites such as whether the test must already exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smartui_baseline_manageSmartUI Baseline ManageADestructiveInspect
Merge SmartUI baselines across branches or builds (to read baseline settings, use smartui_project_query with action=get). merge_branch/merge_build resolve source and target names to builds, merge them, then poll the merged build and report its real terminal status — a merge that lands in error or with zero screenshots is returned as a tool error, not as success. Merging permanently changes baseline state: the merged build takes the baseline flag, so a failed merge leaves the baseline on an empty build.
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | merge_branch merges one branch baseline into another; merge_build merges one build into another | |
| source | No | Source branch (merge_branch) or build name (merge_build) to merge FROM | |
| target | No | Target branch (merge_branch) or build name (merge_build) to merge INTO | |
| project_id | Yes | Project id (required) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the destructiveHint annotation by disclosing the merge lifecycle: names resolve to builds, the merged build is polled, terminal status is reported, and error/zero-screenshot outcomes surface as tool errors rather than success. Critically warns that the merged build takes the baseline flag, so a failed merge leaves the baseline on an empty build — a non-obvious destructive consequence.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with purpose and the read alternative before behavioral detail. Dense but every clause (polling, error semantics, baseline-flag side effect) carries information an agent needs. Slightly packed, but no waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive, multi-mode mutation tool with no output schema, the description covers what the tool does, when to use it vs the read sibling, how success/failure is signaled, and the lasting state change. Nothing an agent needs to call it safely is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (baseline 3), but the description adds meaning: source/target are names that get resolved to builds, and merge_branch vs merge_build reinterpret those same fields as branches or build names. This clarifies the enum-conditional parameter semantics beyond the schema text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (merge) and resource (SmartUI baselines) with scope (across branches or builds), and explicitly distinguishes itself from the read-path sibling smartui_project_query. An agent can identify this as the mutation counterpart to the SmartUI query tools without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes the agent: 'to read baseline settings, use smartui_project_query with action=get', and names the two merge modes (merge_branch/merge_build) with their distinct source/target semantics. When-to-use and the alternative for the read case are both stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smartui_build_querySmartUI Build QueryARead-onlyInspect
List SmartUI builds in a project, get one build, or get the CI gate verdict for a build. Use action=gate in pipelines: it returns verdict (passed|failed|pending_approval|running|error), totals (including missing screenshots the baseline has), blocking_comparison_ids, and blocking[] with per-item detail. verdict=failed covers rejections AND builds that dropped baseline screenshots. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (list only, 1-based) | |
| limit | No | Page size (list only, default 50, max 200) | |
| action | Yes | list builds, get one build, or gate (CI verdict) | |
| branch | No | Filter builds by branch (list only) | |
| build_id | No | Build id — required for get/gate (or use build_name) | |
| build_name | No | Build name. get/gate: exact name, an alternative to build_id. list: case-insensitive substring filter, applied client-side to the newest 200 builds (upstream has no name filter). | |
| project_id | Yes | Project id (required) | |
| build_status | No | Substring filter on build status (list only), e.g. "pending-approval" |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, so the trailing "Read-only" only confirms them. The description earns credit for disclosing semantics annotations cannot: the full verdict enum, that totals include missing baseline screenshots, that blocking[] carries per-item detail, and notably that verdict=failed conflates rejections with dropped baseline screenshots.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the three operations, then the high-value gate return contract, with no filler sentences. Minor waste is repeating "Read-only" already asserted by readOnlyHint=true annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly compensates for the gate path by enumerating verdicts and blocking fields. However, list and get return shapes and pagination behavior go unmentioned, leaving an agent guessing about the other two action paths of this 8-parameter tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, including action enum meanings, build_id/build_name dual usage, and the client-side substring caveat, so the schema already carries parameter semantics. The description only re-expresses action=gate usage and adds no syntax or constraint beyond the schema — baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence names the resource (SmartUI builds) and all three distinct verbs/operations (list, get, gate verdict), so an agent can map action values to intent without opening the schema. The resource itself separates it from smartui_project_query and smartui_comparison_query siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Use action=gate in pipelines" gives an explicit context that selects one of the three actions, which is real routing guidance. It stops short of saying when NOT to use this tool or naming SmartUI siblings (e.g. baseline/comparison tools) as alternatives for related questions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smartui_comparison_analyzeSmartUI Comparison AnalyzeARead-onlyInspect
Analyze one SmartUI comparison through a lens: consolidated (default), pixel, or dom. Returns the comparison data (mismatch %, verdict, correctly-labelled asset URLs) plus, for lens=dom, a real DOM byte delta when the artefact is served — no local filesystem needed. consolidated and pixel report upstream numbers and point at the assets; they do not run their own image analysis. lens=dom degrades to a structured error when DOM artefacts are not served. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| lens | No | Analysis lens (default consolidated). layout and perceptual were retired — they returned the consolidated payload unchanged. | |
| comparison_id | Yes | Comparison id (required) — from smartui_comparison_query | |
| include_assets | No | urls (default) returns asset URLs only. inline additionally attaches the screenshots themselves as image blocks — compared only for lens=pixel, baseline+current+compared for consolidated, none for lens=dom. Images over 1.5 MB (or past 2.25 MB in total) are skipped and listed with the reason; their URLs stay in the result. Use inline when you cannot fetch URLs. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/destructiveHint, and the description reinforces that while adding real behavioral detail beyond them: consolidated/pixel do not run their own image analysis, lens=dom degrades to a structured error when artefacts are absent, and no local filesystem is required. Failure and degradation modes are disclosed, which is exactly the added value the annotation does not carry.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action and lens list, then layers consequences. Dense but every clause carries information; losses come only from mild repetition of the lens behavior in two sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description must describe returns, and it does: mismatch %, verdict, labelled asset URLs, and the DOM byte delta for lens=dom. Combined with the error-degradation note, an agent has enough to call it correctly; only the exact error/response shape remains unspecified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters including the inline size limits, so the baseline is 3. The description mostly restates the lens semantics already present in the schema and adds no new format or syntax guidance for comparison_id or include_assets.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Analyze) and resource (one SmartUI comparison) with an explicit scope limiter (one, not many), and names the three lens modes. It is clearly distinguishable from the sibling smartui_comparison_query, whose output feeds the required comparison_id.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains when each lens matters (dom gives a real byte delta and needs served artefacts; consolidated/pixel merely surface upstream numbers) and gives an actionable condition for include_assets=inline ('when you cannot fetch URLs'). It stops short of stating when to prefer this tool over smartui_comparison_query.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smartui_comparison_querySmartUI Comparison QueryARead-onlyInspect
List all comparisons in a SmartUI build (each row returns comparison_id AND ss_id), or get one comparison's detail (mismatch %, status, correctly-labelled baseline/current/compared asset URLs). This is how you obtain a comparison_id — you never need the dashboard. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (list only, 1-based) | |
| limit | No | Page size (list only, default 50, max 200) | |
| action | Yes | list comparisons in a build, or get one by comparison_id | |
| status | No | Filter by status (list only), e.g. "changes_found", "approved" | |
| build_id | No | Build id — required for action=list | |
| project_id | No | Project id — required for action=list | |
| min_mismatch | No | Only comparisons with mismatch_percentage >= this (list only) | |
| comparison_id | No | Comparison id — required for action=get |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered; the description reinforces this with 'Read-only' and adds real value by enumerating return contents (comparison_id AND ss_id for list; mismatch %, status, labelled baseline/current/compared URLs for get). It does not discuss pagination behavior, which would have helped.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Dense and front-loaded: the primary list/get capability comes first, followed by the key value proposition and a trailing safety note. The standalone 'Read-only.' repeats the annotation, a minor redundancy, but nothing else is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters, no output schema, and only readOnly annotations, the description does the necessary work of summarizing what each action returns. It is nearly complete for agent selection; only pagination/filtering behavior and its relationship to smartui_comparison_analyze are left unstated.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with each of the 8 parameters already documented in the schema including the list-only markers and the list/get enum. The description adds no parameter-level detail (formats, defaults, edge cases) beyond what the schema provides, so the 3 baseline applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific resource (comparisons) with two clear modes (list in a build vs get one), plus the fields each returns. It is easy to distinguish from the other smartui_* siblings by name, but it never explicitly differentiates itself from smartui_comparison_analyze, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It hints at intent with 'This is how you obtain a comparison_id — you never need the dashboard', which tells the agent why to call it. However, it gives no when-not guidance and does not route the agent between this tool and the nearby smartui_comparison_analyze or smartui_build_query alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smartui_project_querySmartUI Project QueryARead-onlyInspect
List SmartUI projects, or get one project with its settings — action=get returns baseline_branch, approvers[], smart_baseline, auto_approval_branch, overwrite_screenshot and project_category alongside platform and project_type. Start here when you do not have a project_id. Read-only.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (list only, 1-based) | |
| limit | No | Page size (list only, default 50, max 200) | |
| action | Yes | list all projects, or get one by project_id | |
| search | No | Case-insensitive substring filter on project name (list only) | |
| project_id | No | Project id — required for action=get |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds non-obvious behavioral detail: exactly which settings fields (baseline_branch, approvers[], smart_baseline, etc.) come back from action=get. That return-shape disclosure is genuine value beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One front-loaded sentence that leads with the list/get split and closes with the routing hint. The field enumeration is slightly long but earns its place given there is no output schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by naming the fields action=get returns, and it flags read-only behavior. Pagination defaults and the full parameter set are left to the schema, which is acceptable since coverage is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so list-only/page/limit/search/project_id semantics are already documented in the schema. The description only echoes action=get's purpose and its returned fields, adding little parameter meaning beyond the structured data. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States specific verbs (list / get) and the resource (SmartUI projects), and enumerates the fields returned by action=get. An agent can distinguish this from sibling smartui_build_query and smartui_baseline_manage without opening any schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly gives a usage cue: 'Start here when you do not have a project_id.' This tells the agent when to reach for this tool first. It does not name an alternative tool or state exclusions, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smartui_screenshot_reviewSmartUI Screenshot ReviewADestructiveInspect
Approve or reject SmartUI comparisons — individually (comparison_ids or ss_ids), or for a whole build (all_matching:{status}). Ids are validated against the build before writing, and results are verified by re-reading status, so the response reports real per-item updated[]/failed[] and whether the baseline was promoted. Approving comparisons on the baseline branch can promote the build to baseline.
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | approve or reject the selected comparisons | |
| ss_ids | No | Screenshot-variant ids to review directly | |
| build_id | Yes | Build id the comparisons belong to (required) | |
| project_id | Yes | Project id (required) | |
| all_matching | No | Review every comparison in the build matching a status | |
| comparison_ids | No | Comparison ids to review (resolved to ss_ids server-side) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only supply readOnlyHint=false and destructiveHint=true; the description adds substantive behavior beyond that: ids are validated against the build before writing, results are verified by re-reading status, the response reports real per-item updated[]/failed[], and approving on the baseline branch can promote the build to baseline. It also discloses that comparison_ids are resolved server-side.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tightly packed sentences with no filler; the operation and its two modes are front-loaded. Slightly dense (multiple ideas per sentence), but every clause carries information an agent needs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates by describing the response shape (per-item updated[]/failed[]) and the baseline-promotion side effect. Combined with full schema coverage of all 6 parameters, an agent has everything required to invoke and interpret the call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds meaning the per-parameter blurbs do not: that comparison_ids and ss_ids are two alternative individual targeting paths, and that all_matching:{status} is a bulk selector over the whole build. The relationship between the targeting parameters is clarified rather than merely restated.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb pair (approve/reject) on a specific resource (SmartUI comparisons) and enumerates the two operating modes (individual ids vs whole-build all_matching:{status}). An agent can distinguish this from smartui_comparison_analyze or smartui_baseline_manage without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly explains when to use each targeting mode: individual comparison_ids/ss_ids versus bulk all_matching by status. It does not name rival siblings (baseline manage, comparison query/analyze) or state exclusions, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
smartui_screenshot_uploadSmartUI Screenshot UploadADestructiveInspect
Upload screenshots to a SmartUI project as a new build, from local file_paths or from source_urls (fetched server-side, so it works without a client filesystem). Pass project_id to auto-resolve the project token. Set mark_baseline=true to seed a baseline. Returns build_id; follow with smartui_build_query(action=gate).
| Name | Required | Description | Default |
|---|---|---|---|
| action | Yes | What to upload (currently images) | |
| build_name | No | Name for this build (optional) | |
| file_paths | No | Absolute local image paths (.png/.jpg/.jpeg). Only usable when the MCP server runs locally over stdio — the hosted server cannot see your filesystem; use source_urls there. | |
| project_id | No | Project id — preferred; the projectToken is resolved automatically | |
| source_urls | No | Public http/https image URLs, fetched server-side (max 20). Works in hosted mode. | |
| mark_baseline | No | Mark this build as the baseline | |
| screenshot_name | No | Name this screenshot (binds it to that baseline) instead of using the file/URL name. Sent exactly as given, with no extension added, so pass the baseline name as it appears in SmartUI (SDK and CLI captures have no extension). Only valid when uploading exactly one image. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and destructiveHint=true, so the agent knows this mutates state. The description adds the build_id return and the gate follow-up, which the annotations do not convey. However, it never explains the destructive side — e.g. that mark_baseline=true may overwrite an existing baseline — which is the most important behavioral caveat for a tool flagged destructive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, correctly front-loaded with the core action and the highest-value caveat (server-side fetch vs client filesystem). Slight redundancy: the 'fetched server-side' note repeats the schema's 'Works in hosted mode'.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly discloses the return value (build_id) and the next step, so an agent can chain into the gate query. The only gap is behavioral — what marking a baseline displaces — which a destructive-flagged mutation tool should mention.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents every parameter, including the stdio-vs-hosted constraint and the 'no extension added' rule for screenshot_name. The description largely restates this, adding only the minor tie between mark_baseline and later screenshot_name binding. Baseline 3 applies when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Upload screenshots to a SmartUI project as a new build') and covers both ingestion modes, file_paths and source_urls. This distinguishes it cleanly from siblings like smartui_screenshot_review and smartui_build_query, which operate on existing builds.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives conditional guidance: file_paths only works over local stdio, source_urls for hosted mode, project_id for token auto-resolution, mark_baseline to seed a baseline. It also names the follow-up tool (smartui_build_query(action=gate)). It stops short of saying when to use this vs smartui_baseline_manage for baseline operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
stopBuildStop Automation BuildADestructiveIdempotentInspect
Stops every running test in a LambdaTest web automation build. Running sessions end immediately.
| Name | Required | Description | Default |
|---|---|---|---|
| buildId | Yes | ID of the build to stop |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, idempotentHint=true, and readOnlyHint=false, so the safety profile is covered. The description adds the concrete blast radius – every running test stops and running sessions end immediately – which is meaningful context for a destructive operation beyond what the annotations state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences with no filler, and the scope-defining clause (every running test in a build) is front-loaded ahead of the effect.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter destructive stop tool with no output schema, the description covers purpose, scope, and immediate effect. It could note error behavior (e.g., already-finished builds) or what is returned, but nothing critical for correct invocation is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There is a single buildId parameter with 100% schema description coverage, so the schema fully documents it. The description adds no format, type, or constraint detail beyond the schema, which is the expected baseline when the schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (stops) and resource (every running test in a LambdaTest web automation build), and scopes it to the whole build rather than a single session, which distinguishes it from sibling stopSession. An agent can select it without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the verb and scope, but the description never states when to prefer this over stopSession or deleteBuild, nor any prerequisite conditions. It is adequate but gives no explicit routing guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
stopSessionStop SessionBDestructiveInspect
Stop an active running test session on LambdaTest.
| Name | Required | Description | Default |
|---|---|---|---|
| sessionId | Yes | The session ID to stop |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and readOnlyHint=false, so the safety profile is covered. The description adds only the state constraint ('active running'), which is useful, but discloses nothing about what happens to the session's data or whether the action is reversible.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One tight sentence with no filler and the action front-loaded. Nothing wastes space, though it is arguably too terse to be maximally useful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter stop operation whose destructive nature is carried by annotations, and with no output schema to explain, the description is nearly sufficient. It only lacks routing against the similar deleteSession/stopScreenshotTest siblings.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With a single parameter at 100% schema description coverage, the schema fully documents sessionId. The description adds no format, source, or example detail beyond what the schema already provides, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Stop') and resource ('active running test session on LambdaTest'), so the agent knows the target and platform. It does not distinguish itself from close siblings like deleteSession or stopScreenshotTest, but the core purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this versus deleteSession (permanent removal) or stopScreenshotTest. The word 'active running' weakly implies the session must be live, but no alternatives or preconditions are named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
stopTunnelStop TunnelADestructiveInspect
Terminate a specific active tunnel connection by tunnel ID. Use this to clean up tunnels that are no longer needed.
| Name | Required | Description | Default |
|---|---|---|---|
| tunnelId | Yes | The ID of the tunnel to stop |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, readOnlyHint=false, and openWorldHint=false, so the safety profile is covered. The description reinforces the terminal nature with 'Terminate' and the cleanup use case, but adds no new behavioral detail such as irreversibility or effect on sessions using the tunnel.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences with the action and key identifier front-loaded, followed by the use case. No filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter destructive tool with full annotation coverage, the description is essentially complete. Only minor omissions remain, such as whether stopping a tunnel affects in-flight sessions, and there is no output schema to require explanation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a single, fully documented parameter, so the schema already carries the meaning. 'by tunnel ID' merely restates what the schema says; baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Terminate) and resource (a specific active tunnel connection) keyed by tunnel ID. It is clearly distinguishable from listTunnels and the analyze/build tunnel tools, though it does not name any sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The second sentence gives a use case ('clean up tunnels that are no longer needed'), which implies when to call it, but there is no explicit when-not guidance or reference to the alternative (e.g., listTunnels to find IDs). Adequate but with clear gaps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_create_folderCreate FolderAInspect
Create a folder — in a project's Test Cases tree (default) or Test Runs tree (pass project_id), or in the organization-level Modules tree (no project_id needed; module folders are shared across projects). Optionally nest it under an existing parent folder to build folder trees. Deleting folders is not available through this connection.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Folder name | |
| tree | No | Which folder tree to create in (default test-cases). "modules" is organization-level. | |
| project_id | No | Project ID (ULID) — required for the test-cases and test-runs trees; not used for modules | |
| description | No | Optional description | |
| parent_folder_id | No | Nest under this existing folder (ULID); omit for top level |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, openWorldHint=false, so the agent knows this is a contained, non-destructive write. The description adds meaningful context beyond that: modules live at organization level and are shared across projects, and deletion is unsupported. It does not say what happens if the name collides or if parent_folder_id is invalid, but the added context is solid.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three compact sentences, front-loaded with the create action and the tree-choice branching, followed by nesting and the deletion caveat. No filler and no restatement of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description covers the branching modes, the required nesting rules, and an important negative capability (no delete). It stops short of explaining return behavior or failure modes for invalid parent/project ids, which is a minor gap given the clean schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real semantics: which tree the default applies to, that project_id is inapplicable for modules, and that omitting parent_folder_id means top level. That is useful reinforcement and clarification rather than pure duplication.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Create) plus resource (folder) and immediately enumerates the three distinct trees it can operate on, with the default called out. This differentiates it cleanly from siblings like tm_list_folders, tm_update_folder, and tm_create_project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states the conditions that select each mode: pass project_id for Test Cases/Test Runs trees, omit it for organization-level Modules, and optionally nest via parent_folder_id. It also sets a boundary by noting that deletion is not available through this connection, which steers the agent away from expecting a delete/undo path.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_create_milestoneCreate MilestoneAInspect
Create a milestone in a project (status: Open), optionally with description, start/end dates, tags, and existing runs attached.
| Name | Required | Description | Default |
|---|---|---|---|
| tags | No | Labels | |
| owner | No | Milestone owner: an organization member (email, username, or user id). Omit for no owner. | |
| title | Yes | Milestone title | |
| end_at | No | End date (YYYY-MM-DD) | |
| start_at | No | Start date (YYYY-MM-DD) | |
| project_id | Yes | Project ID (ULID) | |
| description | No | What this milestone covers | |
| test_run_ids | No | Existing run ids to attach |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, and openWorldHint=false, so the write-but-non-destructive profile is covered. The description adds one genuinely useful behavioral detail — the created milestone defaults to status 'Open' — but says nothing about permissions, whether attached runs must belong to the project, or what the call returns.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single front-loaded sentence stating the core action followed by the optional surface area; no filler and nothing redundant. Appropriate size for a create tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 well-documented parameters, required fields, and annotations covering the safety profile, the description covers enough for correct invocation. It only omits return/ID behavior, which is a minor gap given there is no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (owner, dates, tags, test_run_ids all documented in the schema), so the description's paraphrase of those fields adds little. Baseline 3 is appropriate when the schema carries the parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description gives a specific verb+resource ('Create a milestone in a project') and enumerates the optional payload (description, dates, tags, runs), so it is easily distinguished from siblings like tm_update_milestone and tm_create_project. An agent can identify what the tool produces without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied by the verb 'Create'; there is no explicit when-to-use guidance, no note about prerequisites (e.g., project must exist), and no routing to alternatives such as tm_update_milestone for modifying an existing milestone. Adequate but with clear gaps.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_create_projectCreate ProjectAInspect
Create a new Test Manager project. Project & Org Instructions (Memory Layer) stay managed in the UI. Deleting projects is not available through this connection.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Project name | |
| tags | No | Labels | |
| description | No | What the project covers |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds genuinely non-obvious behavioral context: instructions live in the UI and deletion is unavailable through this connection, which prevents an agent from assuming a full CRUD surface.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the core action, then two tightly-scoped caveats. No filler or restated title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-required-param create tool with annotations covering safety and no output schema, the essentials are present. A minor gap is that it doesn't hint at the result (e.g., whether a project ID is returned), which would matter for chaining into tm_create_folder or tm_create_test_case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage and only three simple params (name, tags, description), the schema carries the parameter meaning. The description adds no format, constraint, or default details beyond it, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Create a new Test Manager project"), which clearly separates it from sibling creation tools like tm_create_folder or tm_create_test_case. It stops short of explicitly contrasting those siblings, but the resource scoping is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The note that "Project & Org Instructions (Memory Layer) stay managed in the UI" implicitly tells the agent not to attempt that here, and the deletion note rules out one adjacent action. However, there is no explicit when-to-use guidance or routing to alternatives such as tm_update_project or tm_create_folder.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_create_test_caseCreate Test CaseAInspect
Create a single test case in a project folder, with optional steps and references to existing step Modules (module steps are included; the Module itself is unchanged). For creating many cases from a requirement, prefer tm_generate_test_cases.
| Name | Required | Description | Default |
|---|---|---|---|
| tags | No | Labels | |
| type | No | e.g. Functional, Regression, Smoke & Sanity | |
| title | Yes | Test case title | |
| status | No | e.g. Live, Unverified, Ready (default per project settings) | |
| priority | No | Lowest, Low, Medium, High or Highest | |
| folder_id | No | Folder to file the case into | |
| module_ids | No | Existing step Module ids to include (appended as module steps after test_steps) | |
| project_id | Yes | Project ID (ULID) | |
| test_steps | No | Ordered steps | |
| description | No | What the case verifies | |
| preconditions | No | Setup required before the steps | |
| estimated_time | No | Estimated execution time in minutes | |
| automation_status | No | Automated, Not Automated or To Be Automated |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the safety profile (readOnlyHint=false, destructiveHint=false, openWorldHint=false), so the bar is lower. The description still adds real behavioral context: module steps are included and appended after test_steps, and critically that 'the Module itself is unchanged' — a non-obvious side-effect clarification the annotations do not convey. It stops short of disclosing what is returned or any permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly written sentences, front-loaded with the create semantics before the routing hint. No filler and no repetition of the tool name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 13-parameter mutation tool with no output schema, the description covers create semantics and the bulk alternative but says nothing about the response (e.g., the created case's id) which the agent will need for follow-up calls. Folder placement when folder_id is omitted and the project-default status behavior are left entirely to the schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the schema already documents all 13 parameters, including the module append-order note and the 'outcome' key constraint. The description adds almost nothing beyond the schema, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a precise verb (Create), resource (a single test case), and scope (in a project folder, with optional steps and module references), and explicitly contrasts itself with the bulk sibling tm_generate_test_cases. An agent can route between the two without opening either schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly names the condition that selects the alternative: 'For creating many cases from a requirement, prefer tm_generate_test_cases.' That is a clear when-to-use-this vs. when-to-use-something-else statement, plus an implicit boundary (single vs. many).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_create_test_runCreate Test RunAInspect
Create a test run from chosen test cases — manual (default) or KaneAI (run_type: "kaneai"). Each included test case becomes a TEST RUN INSTANCE — the executable copy of that test inside the run, which carries its own status, remarks and exactly one Configuration. The cases must match the run type: KaneAI runs only take KaneAI-authored cases with completed code generation; manual runs only take manual cases. Optionally attach existing milestones, file into a run folder, and apply a Configuration to all instances (KaneAI configurations must be compatible with each case's platform — web/mobile-app/mobile-browser and iOS/Android).
| Name | Required | Description | Default |
|---|---|---|---|
| tags | No | Labels | |
| title | Yes | Run title | |
| assignee | No | Assign every created instance to this organization member: email, username, or user id. Omit to leave the instances unassigned. | |
| run_type | No | Run type (default manual). "kaneai" creates a KaneAI run: only KaneAI-authored test cases can be included, and configurations must be kane-supported and platform-compatible. | |
| folder_id | No | Test-run folder to file the run into | |
| objective | No | What this run verifies | |
| project_id | Yes | Project ID (ULID) | |
| is_sequential | No | KaneAI runs only: execute the instances sequentially (default false). | |
| milestone_ids | No | Existing milestone ids to attach the run to | |
| test_case_ids | No | Test case ids to include | |
| configuration_id | No | Apply ONE Configuration to every included test case (the configuration's top-level id from tm_list_configurations) | |
| configuration_ids | No | Apply MULTIPLE Configurations: each included test case gets one test run instance per configuration (case × configuration), like the UI's Add Configuration checkboxes |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, openWorldHint=false, so the safety profile is covered. The description adds real behavioral context beyond that: each case is copied into an executable instance with its own status/remarks/one Configuration, and it exposes a precondition failure mode (mismatched run type, incomplete code generation, incompatible platform).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core action and mode choice, and the TEST RUN INSTANCE definition earns its space because it clarifies the object model. The final sentence packs several constraints (milestones, folder, configuration, platform compatibility) into one dense clause, slightly hurting scanability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 12-parameter create tool with full schema coverage, the description covers run types, prerequisites, instance semantics, and configuration behavior well. With no output schema present, it stops short of describing what the call returns (e.g., the created run/instances), which is the remaining gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description still adds value by explaining the configuration_id vs configuration_ids distinction (one Configuration shared by all instances vs case×configuration expansion) and the KaneAI platform-compatibility rule, neither of which is fully stated in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb+resource ('Create a test run') with the two run modes and their semantic difference spelled out, plus a definition of what an included case becomes (a TEST RUN INSTANCE). No other sibling tool creates a run, so the agent can disambiguate from tm_create_test_case, tm_create_milestone, and tm_generate_test_cases immediately.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context and prerequisites: KaneAI runs only accept KaneAI-authored cases with completed code generation, manual runs only accept manual cases, and options are described as optional (milestones, folder, configuration). It never routes to a named alternative (e.g., when to use tm_generate_test_cases first), so it stops short of explicit when-not guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_generate_test_casesGenerate Test Cases with AIAInspect
Generate structured test cases from a requirement or prompt using the Test Manager AI generator (applies the project's Memory Layer — Project & Org Instructions — automatically) and file them into the chosen folder. Requirement documents, specs, spreadsheets or screenshots can be attached via files to ground the generation. Volume = test_scenario_limit × per_scenario_test_cases_limit, capped at 50 per call. By default all generated cases are saved into the folder (one sub-folder per scenario); pass auto_save=false to review before saving. The call returns FAST with a request_id (and a browser progress URL) — generation takes ~30-90s in the background; call this tool again with just request_id to check progress, fetch results, or save. Note: consumes AI generation credits.
| Name | Required | Description | Default |
|---|---|---|---|
| files | No | Optional reference files to ground the generation (requirement docs, specs, data sheets, screenshots). Each item carries the file name and its raw bytes base64-encoded in content_base64. Supported: .pdf, .docx, .txt, .csv, .xlsx, .json, .xml, .jpg, .jpeg, .png, .gif, .bmp, .webp; max 5 files, 10 MB each. Large files are better uploaded in the Test Manager UI, since the base64 content travels in the request. Only used when starting a new generation. | |
| prompt | No | The requirement / instructions to generate test cases from. Required when starting a new generation. | |
| jira_ids | No | Optional Jira issue keys to use as requirement context | |
| auto_save | No | Save all generated cases into the folder automatically (default true for a NEW generation). Set false to review first — that choice persists across request_id polls (a bare poll never saves it); to save after review, call again with request_id AND auto_save=true. | |
| folder_id | No | Folder to file generated cases into. Required when starting a new generation. | |
| project_id | No | Project ID (ULID). Required when starting a new generation. | |
| request_id | No | Resume/act on an earlier generation: check progress, fetch results, or save them. | |
| test_scenario_limit | No | Max scenarios to generate (default 3) | |
| per_scenario_test_cases_limit | No | Max test cases per scenario (default 3) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=false and destructiveHint=false; the description adds the async contract (returns FAST with request_id + progress URL, ~30-90s background), the credit consumption, automatic Memory Layer application, and the 50-case cap. These are behavioral traits the agent cannot get from structured fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The purpose and key constraints (files, volume cap, auto_save default, async flow, credits) are front-loaded and nearly every sentence carries actionable detail. It is dense but not padded, though the parenthetical Memory Layer note slightly interrupts flow.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an async, no-output-schema, 9-parameter generation tool, the description covers the return value (request_id + progress URL), the two-phase workflow, saving behavior, and cost — everything an agent needs to invoke and follow up correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented, but the description adds real semantics: the volume formula (test_scenario_limit × per_scenario_test_cases_limit, cap 50), and the request_id resume/poll behavior including that the auto_save=false choice persists across polls.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource ('Generate structured test cases from a requirement') and names the generator and destination folder. It is clearly distinguishable from the manual sibling tm_create_test_case by emphasizing AI generation from a prompt, not hand-authored creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly covers when to use which mode: attach files to ground generation, pass auto_save=false to review before saving, and re-call with just request_id to check progress/fetch/save. It also warns that generation consumes AI credits, which is a genuine when-to-use consideration.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_get_coverage_summaryGet Coverage SummaryARead-onlyInspect
Test coverage for a project, overall and per folder, computed live from the project's test cases (matches the Test Cases screen). Also answers requirement and traceability coverage: pass jira_id (any Jira issue key, e.g. TE-123: a requirement, user story, epic, ticket or bug) to get that item's test coverage (the test cases linked to it and how many are automated; zero linked cases means uncovered). For a set of requirements, call once per Jira key. A scope with no test cases returns all zeros; a Jira key that does not exist in the connected Jira returns an error.
| Name | Required | Description | Default |
|---|---|---|---|
| jira_id | No | Optional Jira issue key (e.g. TE-123) — scope coverage to the test cases linked to this requirement / user story / epic / ticket (traceability coverage) | |
| folder_id | No | Optional folder scope | |
| project_id | Yes | Project ID (ULID) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=false, so the safety profile is covered. The description adds genuine behavioral context beyond that: coverage is computed live from test cases, a scope with no test cases returns all zeros, and a nonexistent Jira key returns an error rather than an empty result.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single dense paragraph with the core purpose front-loaded, followed by the jira_id mode and edge-case behavior. Every sentence carries information, though the middle clause enumerating Jira issue types is slightly over-packed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly compensates by describing what comes back (per-folder and overall coverage, zeros for empty scope, error for unknown Jira keys) and how automation counts factor in. It stops short of describing the actual response shape or field names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3, but the description adds real semantics the schema doesn't carry: jira_id accepts any Jira issue type (requirement, user story, epic, ticket, bug) and zero linked cases explicitly means uncovered. folder_id and project_id get no extra detail, so it does not reach 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (test coverage for a project, overall and per folder) plus a second mode (requirement/traceability coverage via jira_id), and anchors it to the Test Cases screen. No sibling tool computes coverage, so the distinction is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit routing: pass jira_id for requirement/traceability coverage, and 'For a set of requirements, call once per Jira key' tells the agent how to batch. It lacks an explicit 'when not to use this' or named sibling alternative, keeping it short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_get_milestoneGet MilestonesARead-onlyInspect
Pass milestone_id to get one milestone's detail and live completion progress. Otherwise pass project_id to list the project's milestones.
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | No | Project ID (ULID). Required when milestone_id is not given. | |
| milestone_id | No | Milestone ID (ULID). When given, returns detail + progress. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and closed-world scope, so the safety profile is covered. The description adds only the note that detail mode returns 'live completion progress', which is mild extra context. No mention of ordering, pagination, or empty-result behavior for the list mode.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no filler, and the distinguishing condition (milestone_id vs project_id) is front-loaded in the first clause. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter read-only tool with full schema coverage and safety annotations, the description covers the essentials an agent needs to choose a call path. It is slightly thin on list-mode behavior, but nothing critical is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and both parameters already document ULID format, requiredness, and conditional behavior. The description restates the same either/or logic rather than adding new meaning, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource (get milestone detail vs list milestones) and clearly separates the two modes by which parameter is supplied. It does not name or differentiate against any sibling tool, but the dual-mode wording is unambiguous about what comes back.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives an explicit conditional: pass milestone_id for one milestone, otherwise pass project_id to list. That is clear routing logic for a single tool. It stops short of naming alternatives (e.g., tm_list_projects) or stating exclusions, so it does not reach 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_get_test_casesFind / Get Test CasesARead-onlyInspect
Find test cases in a project by filters (folder, priority, automation status, type, tags) and/or search (matches test case ID like TC-123, or title; minimum 3 characters) — returns summaries. Filters combine (AND). Pass case_id instead to get one case's full detail (steps, BDD scenarios, field values).
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-based) | |
| tags | No | Filter by tags | |
| type | No | Filter by test type (e.g. Functional, Regression, Smoke & Sanity) | |
| limit | No | Max results per page | |
| search | No | Full-text keyword (min 3 chars) | |
| case_id | No | Test case ID (ULID). When given, returns that case's full detail and all other filters are ignored. | |
| priority | No | Filter by priority (e.g. Lowest, Low, Medium, High, Highest). A value outside the project's priority list is rejected with the valid values. | |
| folder_id | No | Only cases in this folder | |
| project_id | No | Project ID (ULID). Required unless case_id is given. | |
| automation_status | No | Filter by automation status (e.g. Automated, Not Automated, To Be Automated) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, so the safety profile is covered. The description still adds real behavioral context: results are summaries, the search must be at least 3 characters, filters combine via AND, and case_id silently overrides all other filters. That override behavior is a genuinely useful warning the annotations do not convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences, front-loaded with the primary search use case before the single-case alternative. Every clause carries information; no filler or restatement of the title.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description partly compensates by naming the return shape (summaries vs. steps/BDD scenarios/field values). It omits any mention of pagination behavior despite page/limit parameters, which is the main remaining gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by clarifying what `search` matches (test case ID like TC-123 or title), the AND semantics of filters, and that case_id causes other filters to be ignored — semantics the schema only partially states.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the specific resource (test cases in a project) and the two distinct modes: filtered search returning summaries, and single-case lookup via case_id returning full detail. That distinction lets an agent tell this apart from the sibling list/search tools without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit routing within the tool ('Pass case_id instead to get one case's full detail') and states that filters combine with AND. It does not name a sibling tool (e.g. tm_get_test_runs, tm_list_folders) as an alternative, so it stops short of the 5 rubric's explicit alternative naming.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_get_test_runsGet Test RunsARead-onlyInspect
Pass run_id to get one test run's detail plus its result summary (instance counts by status — passed / failed / skipped / not started / in progress — and pass rate). Otherwise pass project_id to list runs with optional filters. Every returned run carries run_type ("kaneai" or "manual") identifying KaneAI runs — the backend stores KaneAI runs with type "Manual" (result-entry mode), so use run_type, not type, to tell them apart.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-based) | |
| limit | No | Max results per page | |
| run_id | No | Test run ID (ULID). When given, returns detail + result summary. | |
| search | No | Search runs by title | |
| status | No | Filter: Not Started, In Progress, Passed, Failed, Skipped | |
| run_type | No | Filter: "kaneai" (KaneAI runs), "manual" (human result-entry runs, KaneAI excluded), or "automation" (framework-uploaded runs). Buckets are disjoint. | |
| folder_id | No | Only runs in this folder | |
| project_id | No | Project ID (ULID). Required when run_id is not given. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so safety is covered. The description does add genuinely non-obvious context about the backend storing KaneAI runs as type "Manual" and instructing use of run_type instead of type, which is valuable. It stops short of covering pagination behavior for the list mode.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three dense sentences with no filler. The primary routing rule is front-loaded, and the caveat about run_type is placed last where it matters most for interpretation.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description shoulders the return-value burden and does describe the detail payload (instance counts by status, pass rate). Pagination semantics for the list branch (page/limit) are left to the schema, a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description goes beyond the schema by clarifying the run_id-vs-project_id dependency and by warning that run_type, not the backend's type field, is the reliable discriminator — real added meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource and distinguishes the two operating modes: run_id yields a single run's detail plus result summary, while project_id lists runs with filters. An agent knows exactly what it will get without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly routes usage: pass run_id for one run, otherwise pass project_id to list. The alternative branch and the selecting condition are both stated, so no inference is required.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_link_ado_issueLink Azure DevOps Work ItemAInspect
Link a test case, test run, or a run result (instance) to an Azure DevOps work item (requirement / user story / bug) by its URL — linked test cases are what give the work item its test coverage and traceability. Requires the Azure DevOps integration to be set up for the organization — otherwise a clear "set up Azure DevOps first" message is returned.
| Name | Required | Description | Default |
|---|---|---|---|
| entity_id | Yes | Id of the case (ULID), run (ULID) or instance (number as string) | |
| project_id | Yes | Project ID (ULID) the entity belongs to | |
| entity_type | Yes | What to link | |
| work_item_url | Yes | The Azure DevOps work item URL, e.g. https://dev.azure.com/<organization>/<project>/_workitems/edit/165 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare a non-read-only, non-destructive, closed-world mutation, so the description doesn't need to restate safety. It adds real value by disclosing the setup dependency and that a clear "set up Azure DevOps first" message is returned when integration is missing, plus the downstream effect on coverage/traceability.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two dense sentences that front-load the action and the entity/target pair, followed by the prerequisite. The em-dash clause about coverage and traceability is slightly promotional but earns its place by explaining why the call matters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description compensates by describing the failure mode (integration-missing message) and the semantic outcome of linking. With all four params documented and annotations covering safety, this is sufficient for correct invocation, though the absence of any note on permission/auth requirements beyond setup is a minor gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all four parameters (entity_id, project_id, entity_type, work_item_url) are already documented in the schema, including the URL format example. The description adds only high-level meaning (which entity types exist, what a work item represents), which is baseline-level value given the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (link) and precise resource set — test case, test run, or run instance — connected to an Azure DevOps work item identified by URL. It names the target domain (ADO) and the work item kinds (requirement/user story/bug), making it easily distinguishable from the sibling tm_link_jira_issue.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context for use — linking is what produces test coverage and traceability — and states a hard prerequisite (ADO integration must be configured for the organization). However, it never explicitly names the alternative (tm_link_jira_issue) or an exclusion condition to route the agent between the two linking tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_link_jira_issueLink Jira IssueAInspect
Link a test case, test run, or a run result (instance) to a Jira issue (requirement / user story / epic / bug) — linked test cases are what give the issue its test coverage and traceability. Linking a result also links its case and run. Requires the Jira integration to be set up for the organization — otherwise a clear "set up Jira first" message is returned.
| Name | Required | Description | Default |
|---|---|---|---|
| issue_id | Yes | Jira issue key, e.g. TE-123 | |
| entity_id | Yes | Id of the case (ULID), run (ULID) or instance (number as string) | |
| entity_type | Yes | What to link |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=false, destructiveHint=false, and openWorldHint=false, so the safety profile is mostly covered. The description adds non-obvious behavioral context beyond that: linking a result cascades to link its case and run, and it discloses the prerequisite-failure behavior (a clear 'set up Jira first' message).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose before the cascade caveat and the prerequisite. Slightly dense but every clause adds meaningful information; no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema, the description covers the key behaviors an agent needs: what gets linked, the cascade to case/run, and the failure mode when Jira isn't configured. It omits return-shape details, but those are not essential for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with all three parameters documented, so the baseline is 3. The description adds marginal value by clarifying the kinds of Jira issues (requirement/user story/epic/bug) and the entity types, but the ULID/number formats and enum values are already fully specified in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (link) and the exact resources involved (test case, test run, or run result to a Jira issue), and explains the downstream value (test coverage, traceability). This clearly distinguishes it from the sibling tm_link_ado_issue, which links to a different tracker.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a clear usage context (linking is what creates coverage/traceability) and an explicit precondition: the Jira integration must be set up, otherwise a 'set up Jira first' message is returned. It does not explicitly name the ADO sibling as the alternative path, which keeps it short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_list_configurationsList Run ConfigurationsARead-onlyInspect
List saved run Configurations (reusable platform / browser / OS / device setups). To apply one to a run, pass the configuration's TOP-LEVEL id to tm_create_test_run or tm_update_test_run as configuration_id (NOT the nested environments[].environment_id). Pass run_type to see only the configurations usable for that run type (kaneai vs manual). Each test instance carries exactly one configuration. Configurations are created by humans in the Test Manager UI. They are organization-level (shared by every project), so there is no project filter. Results are paged: the response says how many exist in total; page through them or raise limit to see all.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-based) | |
| limit | No | Max results per page (default 10) | |
| run_type | No | Only configurations supporting this run type (kane-supported vs manual-supported) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, and the description adds substantial context beyond that: configurations are human-created in the UI (not via API), are organization-level and shared across all projects, and results are paged with a total count in the response. These are non-obvious behavioral traits an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core purpose, then dense, high-value sentences on the id distinction and paging. Slightly long, but nearly every sentence carries actionable information; the 'each test instance carries exactly one configuration' line is marginally less essential.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still covers return behavior (total count, pagination) and the org-level scope constraint. Combined with annotations covering the safety profile, an agent has everything needed to call this correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds real semantic value: it warns that the TOP-LEVEL id (not nested environments[].environment_id) must be used as configuration_id, and clarifies run_type's kaneai vs manual meaning. Page/limit are only referenced generically ('page through them or raise limit').
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (List) and resource (saved run Configurations) and defines what a configuration is (reusable platform/browser/OS/device setups). It also distinguishes itself from the sibling tools tm_create_test_run and tm_update_test_run by explaining that this tool is the source of the configuration_id those tools consume.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context: pass run_type to narrow by kaneai vs manual, and notes there is no project filter because configurations are org-level. It routes the agent to sibling tools for applying a configuration but doesn't state explicit when-not-to-use conditions beyond the implicit 'use tm_create_test_run to apply'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_list_foldersList FoldersARead-onlyInspect
Get a folder tree. Test Manager keeps three folder types: Test Cases and Test Runs folders (per project — pass project_id) and Modules folders (organization-level, since modules are shared across projects — no project_id needed). Pick with the tree parameter (default: test-cases). New test-case/test-run folders can be created with tm_create_folder.
| Name | Required | Description | Default |
|---|---|---|---|
| tree | No | Which folder tree to return (default test-cases). "modules" is organization-level. | |
| project_id | No | Project ID (ULID) — required for the test-cases and test-runs trees; not used for modules |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint/openWorldHint/destructiveHint, so safety is covered. The description adds real data-model behavior beyond that: modules are organization-level and shared across projects while test-case/test-run folders are project-scoped. It does not describe return shape or pagination, but no output schema exists to carry that either.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tightly written sentences with the core action front-loaded, followed by the type taxonomy and the parameter rule. No filler and nothing repeated that the schema already says at greater length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-param read tool with full schema coverage and safety annotations, the description supplies everything needed to choose the right tree and parameters. The only gap is return format/pagination, which is minor given there is no output schema to align with.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters and the enum are already documented in the schema, making this the baseline case. The description restates the tree default and the tree/project_id coupling, which is mildly reinforcing but largely duplicative of the schema text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States the specific verb and resource ('Get a folder tree') and immediately enumerates the three folder types the Test Manager model contains, including which are per-project vs organization-level. An agent can distinguish this from tm_list_modules and tm_list_projects without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear context: pass project_id for test-cases/test-runs, omit it for modules, and select via the tree parameter (default test-cases). It also routes creation work to tm_create_folder. It stops short of an explicit 'do not use this for X' exclusion, but coverage is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_list_modulesList Step ModulesARead-onlyInspect
List the organization's reusable step Modules (modules are org-level, shared across projects — the same list as the Modules page). Supports name search and pagination. Reference a module by id when creating or editing a test case to include its steps (the Module itself is never changed).
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-based) | |
| limit | No | Max results per page (default 100) | |
| search | No | Filter modules by name keyword | |
| folder_id | No | Only modules in this Modules folder (folder ids from tm_list_folders with tree="modules") | |
| include_subfolders | No | With folder_id: also include modules from its subfolders |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly/non-destructive/non-open-world, so safety is covered. The description adds genuinely useful context beyond that: modules are org-level and shared, the list mirrors the Modules page, pagination is supported, and referencing a module never mutates it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences with purpose front-loaded and the cross-reference to test-case creation placed last. Slightly padded by the parenthetical restating the Modules-page equivalence, but nothing is confusing or out of order.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, and the description conveys enough for correct invocation: read-only listing, org scope, search/pagination support, and how ids are consumed. Minor gaps (return shape, default page size) are covered by the schema's parameter docs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all five parameters (page, limit, search, folder_id, include_subfolders) are already documented. The description only mentions search and pagination generically, adding no syntax or semantics beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Specific verb + resource ('List the organization's reusable step Modules') with scope defined (org-level, shared across projects, equivalent to the Modules page). No sibling tool lists modules, and the description makes clear this is distinct from folders/projects/test cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives a concrete downstream use case: reference a module by id when creating or editing a test case. It does not state when NOT to use it or name an alternative, but there is no competing module-listing sibling, so the guidance is clear if not exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_list_projectsList Test Manager ProjectsBRead-onlyInspect
List all Test Manager projects the authenticated user is allowed to see. Supports keyword search and pagination.
| Name | Required | Description | Default |
|---|---|---|---|
| page | No | Page number (1-based) | |
| limit | No | Max results per page (default 25) | |
| search | No | Filter projects by name keyword |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds the meaningful access-control nuance that results are limited to what the user is permitted to see. It says nothing about ordering, pagination limits, or result shape beyond what the schema states.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences that front-load the core scope before the optional capabilities. No filler or repetition; only slightly short of ideal since the second sentence is a bare feature list.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a simple read-only list tool with full schema coverage and clear annotations, and no output schema is needed. The description covers scope, filtering, and pagination adequately; only guidance about defaults and result ordering is absent, which is minor.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so page, limit, and search are fully documented in the schema. The description's mention of keyword search and pagination merely restates those parameters without adding format or edge-case detail, so the baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('List all Test Manager projects') plus an authorization scope ('the authenticated user is allowed to see'). An agent can distinguish it from write-oriented siblings like tm_create_project or tm_update_project, though it is not differentiated from other tm_list_* readers such as tm_list_folders or tm_list_modules.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description identifies supported capabilities ('keyword search and pagination') but gives no guidance on when to choose this tool over alternatives such as tm_get_test_runs or tm_list_folders, nor any prerequisites or exclusions. Usage is only inferable from the tool's name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_record_test_resultsRecord Test Results (Bulk)ADestructiveInspect
Record execution results on a MANUAL run's TEST RUN INSTANCES (the per-run copies of each test case) — max 500 per request, all-or-nothing (larger requests are rejected with nothing recorded). Manual runs only: KaneAI run results are recorded by KaneAI executions, so manual entry on a KaneAI run is rejected. Target an instance by instance_id, or just pass test_case_id and the instance is resolved automatically. Valid statuses: Passed, Failed, Skipped, Not Started (terminal — run-level "In Progress" is set via tm_update_test_run). Re-recording a step status replaces its result: an omitted step remark is cleared; response echoes per-step outcomes. Optional remarks and assignee per result, and per-step results via steps[] (mark individual test steps / BDD scenario rows Passed/Failed/Skipped with remarks, like the UI's per-step Mark Status). Note: this sets INSTANCE statuses; the run's overall status is set separately via tm_update_test_run.
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Test run ID (ULID) | |
| results | Yes | Results to record (max 500) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the destructiveHint=true / readOnlyHint=false annotations: max 500 per request, all-or-nothing rejection, terminal-only instance statuses, and the non-obvious replacement behavior (re-recording a step clears an omitted remark, response echoes per-step outcomes). This is exactly the operational detail an agent needs before mutating results.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loads the core purpose and is dense with useful content, but the run-vs-instance status note is stated twice (once parenthetically in the status list and again as a closing 'Note'), which is mild redundancy in an otherwise tightly packed paragraph.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, yet the description still tells the agent the response echoes per-step outcomes. Combined with batch limits, failure semantics, status validity, and the run-level hand-off, nothing an agent needs to invoke this mutating tool correctly is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds genuine resolution semantics: instance_id takes precedence while test_case_id triggers automatic resolution, plus the valid status set and the all-or-nothing constraint on the results array. It stops short of detailing remarks/assignee formats, which the schema already carries.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource — recording execution results on test run instances — and immediately narrows scope to MANUAL runs' per-run copies. It also distinguishes itself from the run-level sibling (tm_update_test_run) and from KaneAI executions, so an agent can tell it apart without opening a schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when the tool applies (manual runs, targeting by instance_id or auto-resolved test_case_id) and when it does not ('KaneAI run results are recorded by KaneAI executions, so manual entry on a KaneAI run is rejected'). It names the alternative for run-level status changes, tm_update_test_run, and clarifies that this tool only sets instance statuses.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_update_folderUpdate FolderADestructiveInspect
Rename a folder, change its description, or move it under a different parent folder. Works on all three folder types: per-project Test Cases (default) and Test Runs trees, and the organization-level Modules tree. Moves of test-cases/test-runs folders require project_id; module folders need none.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | New name | |
| tree | No | Which folder tree the folder belongs to (default test-cases). "modules" is organization-level. | |
| folder_id | Yes | Folder ID (ULID) | |
| project_id | No | Project ID (ULID) — required when moving a test-cases or test-runs folder; not used for modules | |
| description | No | New description | |
| parent_folder_id | No | Move under this folder (ULID), or pass "root" to make it a top-level folder; cannot be the folder itself or one of its descendants |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and readOnlyHint=false, so the mutation risk is carried by structured data. The description usefully adds that moves have tree-dependent project_id requirements, but says nothing beyond the schema about reversibility or what happens to existing children.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the three supported operations before the tree-scope constraints. Every clause earns its place with no filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool whose safety profile and parent constraints are covered by annotations and schema, the description supplies the critical tree-type/project_id conditional. Minor gaps like uniqueness or failure behavior remain but are not essential to invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds genuine semantic value by clarifying the conditional relationship between tree type and project_id that the flat schema cannot express. It reinforces rather than merely repeats the parameter docs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States specific verbs and resource: rename, change description, move, on folders. Distinguishes the three folder-tree scopes (Test Cases, Test Runs, Modules) so the agent understands the tool's reach versus siblings like tm_create_folder or tm_list_folders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains the conditional context: test-cases/test-runs moves require project_id while module folders need none, and 'modules' is organization-level. Clear usage context, though it doesn't explicitly name an alternative tool for editing non-folder entities.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_update_milestoneUpdate MilestoneADestructiveInspect
Update a milestone — title, description, dates, tags — or change its status (e.g. mark it Completed). Deleting milestones is not available through this connection.
| Name | Required | Description | Default |
|---|---|---|---|
| tags | No | Replace tags | |
| owner | No | Change the milestone Owner — an organization member (email, username, or user id). | |
| title | No | New title | |
| end_at | No | End date (YYYY-MM-DD) | |
| status | No | New status: Open (reopen) or Complete — the UI's two milestone states. "Active" and "Completed" are accepted as aliases and normalized to Open/Complete (any other stored value hides the milestone from the UI lists). | |
| start_at | No | Start date (YYYY-MM-DD) | |
| project_id | Yes | Project ID (ULID) | |
| description | No | New description | |
| milestone_id | Yes | Milestone ID (ULID) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and destructiveHint=true, so the write/overwrite risk profile is covered. The description adds a genuine constraint (no deletion path) but omits update semantics that matter for a destructive tool — whether omitted fields are left unchanged, whether tags replace or append, and any permission requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, capability first and the exclusion second, both front-loaded with no filler. There is slight redundancy in re-listing fields that the schema already enumerates, which keeps it just short of ideal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 9-parameter mutation tool with full schema coverage, no output schema, and annotations covering the safety profile, the description is nearly sufficient. The missing piece is partial-update semantics and required-permission context, which an agent would want before issuing a destructiveHint=true call.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and includes richer detail than the description (status alias normalization to Open/Complete, ULID formats, tags replacement, owner resolution). The description only restates the field list, adding no parameter meaning beyond the schema, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Update) plus resource (milestone) and enumerates the mutable surface — title, description, dates, tags, status. It does not name or contrast with sibling tools (tm_get_milestone, tm_create_milestone, tm_update_project), so an agent must rely on the tool name alone to disambiguate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives one useful boundary — 'Deleting milestones is not available through this connection' — which rules out an adjacent intent, and the status example implies a reopen/complete workflow. However, there is no guidance on when to prefer this over tm_update_project or how to discover the milestone_id (via tm_list_projects/tm_get_milestone) first.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_update_projectUpdate ProjectADestructiveInspect
Update a project's name, description or tags (fields are merged — only what you pass changes). Project & Org Instructions stay managed in the UI.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | New name | |
| tags | No | Replace tags | |
| project_id | Yes | Project ID (ULID) | |
| description | No | New description |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare destructiveHint=true and readOnlyHint=false. The description adds valuable behavioral context: patch-style merge semantics and that instructions cannot be updated here. It does not cover permissions, error behavior, or response shape, but those gaps are less critical given the safety annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two tight sentences with no filler. It front-loads the core action and then immediately adds the key merge and restriction details, both of which earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple update tool with full schema coverage, safety annotations, and no output schema, the description covers the essential scope and merge behavior. It could mention that project_id is required or describe error cases, but those are minor given the structured fields already available.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameter meaning is already fully documented. The description restates the updatable fields and adds merge semantics, but it does not add param-specific syntax or constraints beyond what the schema provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description names a specific verb and resource: update a project's name, description or tags. This clearly distinguishes it from siblings like tm_create_project and tm_update_folder. It also signals the update scope, so an agent can identify the tool without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
It states that fields are merged and only passed fields change, which tells the agent how to invoke it. It also clarifies that Project & Org Instructions remain UI-managed, giving a useful when-not boundary. However, it does not explicitly name alternative tools or elaborate on when to choose this over other update operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_update_test_caseUpdate Test CaseADestructiveInspect
Edit an existing test case (fields are merged — only what you pass changes). test_steps, when given, REPLACE the case's steps; module_ids append module reference steps. Deleting test cases is not available through this connection.
| Name | Required | Description | Default |
|---|---|---|---|
| tags | No | Replace tags | |
| type | No | Test type | |
| title | No | New title | |
| status | No | e.g. Live, Archived, Faulty, Ready | |
| case_id | Yes | Test case ID (ULID) | |
| priority | No | Lowest, Low, Medium, High or Highest | |
| folder_id | No | Move to this folder | |
| module_ids | No | Existing step Module ids to append as module steps | |
| project_id | Yes | Project ID (ULID) | |
| test_steps | No | Replacement steps (omit to keep current steps) | |
| description | No | New description | |
| preconditions | No | New preconditions | |
| estimated_time | No | Estimated minutes | |
| automation_status | No | Automated, Not Automated or To Be Automated |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=true and readOnlyHint=false; the description earns credit by specifying exactly what is destroyed ('test_steps, when given, REPLACE the case's steps') and by disclosing the merge invariant for all other fields. This is precisely the additive context the destructive hint cannot convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences with zero filler; the merge rule, the replace/append distinction, and the deletion exclusion are all front-loaded before any field-specific detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter mutation tool with no output schema, it covers the highest-risk behaviors (partial update, step replacement, no delete). It omits permission/auth requirements and whether module_ids appends interact with test_steps replacement, which would be the next most useful detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description goes slightly beyond by stating a tool-wide merge rule that governs every optional parameter, and by contrasting replace (test_steps) with append (module_ids) semantics that would otherwise have to be inferred field-by-field.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Edit an existing test case') and immediately qualifies the semantics with 'fields are merged — only what you pass changes'. It implicitly contrasts with tm_create_test_case/tm_update_test_run by saying 'existing', though it never names a sibling explicitly.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives clear operational guidance: partial updates are safe (merge), test_steps replaces rather than appends, module_ids appends, and deletion is explicitly unavailable 'through this connection'. It rules out the delete use case but does not name the alternative tool or surface for deletion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
tm_update_test_runUpdate Test RunADestructiveInspect
Update an open test run — overall status (Passed / Failed / Skipped / In Progress), title, objective, milestones, folder, add test cases (each added case becomes a new test run instance; cases must match the run's type — KaneAI or manual), apply a Configuration to the run's instances (configuration_id; on KaneAI runs it must be kane-supported and platform-compatible with the targeted cases), toggle sequential execution (KaneAI runs), or archive it. Only open (active) runs can be updated; a closed/archived run returns a clear message. Adding cases/milestones appends, never replaces.
| Name | Required | Description | Default |
|---|---|---|---|
| title | No | New title | |
| run_id | Yes | Test run ID (ULID) | |
| status | No | Set the run's OVERALL status (independent of per-instance results). Manual runs only — KaneAI run status is managed by KaneAI executions. | |
| archive | No | Archive (close) the run | |
| assignee | No | Assign the run's instances to this organization member: email, username, or user id ("none" clears). Scope with apply_to_test_case_ids. Omit to keep the current assignee. | |
| folder_id | No | Move to this run folder | |
| objective | No | New objective | |
| is_sequential | No | KaneAI runs only: execute the instances sequentially. | |
| milestone_ids | No | Milestone ids to attach (appended) | |
| configuration_id | No | Set ONE Configuration on the run's instances (the configuration's top-level id from tm_list_configurations; each instance carries exactly one configuration) | |
| add_test_case_ids | No | Test case ids to add to the run (appended) | |
| configuration_ids | No | APPEND multiple Configurations: each targeted test case gets an additional instance per configuration (case × configuration, like the UI); existing configured instances are untouched | |
| apply_to_test_case_ids | No | With configuration_id or configuration_ids: only apply to these test cases (default: all instances in the run) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare destructiveHint=true but not what is destroyed; the description discloses that archive closes the run and, importantly, that adds are append-only ('Adding cases/milestones appends, never replaces') and that each added case spawns a new run instance. It does not mention permissions/auth or reversibility of archive. No contradiction with the readOnlyHint=false / destructiveHint=true annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the core action, then a dense but organized enumeration of mutatable fields with their constraints. It is long for one paragraph and relies on semicolons, but for a 13-parameter tool with several interacting modes almost every clause carries operational information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description covers the exception path ('a closed/archived run returns a clear message') and the append-vs-replace behavior an agent must anticipate. Auth requirements and the exact behavior of configuration_ids with apply_to_test_case_ids defaults could be clearer, but the definition is otherwise self-sufficient for a complex update tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description adds cross-parameter meaning the schema alone does not: configuration_id replaces vs configuration_ids appends, apply_to_test_case_ids scopes both, and configuration_id must be kane-supported and platform-compatible on KaneAI runs. That is real semantic value beyond the per-field text.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Opens with a specific verb+resource ('Update an open test run') and then enumerates the exact facets that can be changed (status, title, objective, milestones, folder, cases, configuration, sequential, archive). An agent can immediately separate this from tm_create_test_run, tm_get_test_runs and tm_update_test_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives concrete preconditions: 'Only open (active) runs can be updated; a closed/archived run returns a clear message', plus field-level gating (status is manual-only, is_sequential is KaneAI-only, cases must match the run's type). It stops short of naming a sibling alternative for related write paths (e.g. tm_record_test_results), but the when-conditions are explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
triggerSuiteRunTrigger Suite RunAInspect
Triggers a phone call test run for a specific suite. This will initiate phone calls for all scenarios configured in the suite.
| Name | Required | Description | Default |
|---|---|---|---|
| projectId | Yes | The project ID that contains the suite | |
| suiteName | Yes | The name of the suite to trigger |
Output Schema
| Name | Required | Description |
|---|---|---|
| data | No | Call initiation results |
| message | No | Status message |
| success | No | Whether the call initiation was successful |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, openWorldHint=true, destructiveHint=false, and the description adds genuinely new behavioral context: this initiates real outbound phone calls for every configured scenario, a costly physical-world side effect. It still omits async behavior, whether the run can be cancelled/stopped, and whether it is idempotent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler, with the action and its blast radius ('all scenarios') front-loaded. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a trigger tool with a 2-parameter fully documented schema and an output schema that covers returns, the description is nearly sufficient. The remaining gap is operational: it does not say how to track the launched run or whether the call is asynchronous, which matters for a long-running phone-call test.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both required parameters (projectId, suiteName) fully documented in the schema itself. The description adds no syntax, format, or naming-convention detail beyond what the schema already provides, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb plus resource ('Triggers a phone call test run for a specific suite') and clarifies scope by noting it fires every scenario in the suite. No sibling is named, but no other tool in the list triggers a suite run, so confusion risk is low.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the effect but never says when to use it versus alternatives (e.g. getSuiteOverview for inspecting a suite, or how to observe the resulting run). No prerequisites, permissions, or preconditions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
updateBuildRename Automation BuildBDestructiveIdempotentInspect
Renames a LambdaTest web automation build. Overwrites the current build name.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | New build name | |
| buildId | Yes | ID of the build to rename |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, idempotentHint=true, and readOnlyHint=false, so the safety profile is covered. The description adds the useful detail that the existing name is overwritten (the old value is lost), but says nothing about permissions, error cases, or whether buildId must reference an existing build.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the action and followed by the one behavioral caveat. Nothing is wasted and nothing is buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter mutation with full schema coverage and annotations declaring destructiveness and idempotency, the description is nearly sufficient. It lacks only error/permission context, which is minor here.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both 'name' and 'buildId' documented inline, so the schema carries the burden. The description adds no format, constraint, or lookup detail beyond that, which is the expected baseline when the schema is complete.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: 'Renames a LambdaTest web automation build.' An agent can distinguish this from deleteBuild, stopBuild, and updateSession without inspecting schemas, though it does not explicitly name which sibling to prefer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no prerequisites, and no mention of alternatives such as deleteBuild or updateSession. The agent must infer the trigger context entirely.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
updateSessionUpdate SessionADestructiveInspect
Update the name or pass/fail status of a test session. Use status_ind to mark tests as passed, failed, error, or skipped from your test framework.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | New name for the session | |
| sessionId | Yes | The session ID to update | |
| status_ind | No | Test result status |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and destructiveHint=true, so the safety profile is covered structurally. The description adds the status_ind semantics but says nothing about what the update overwrites, reversibility, or auth requirements, so it contributes only marginal context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tight sentences that front-load the resource and the two updatable fields, with no wasted verbiage. Slightly terse rather than maximally informative, but well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a low-complexity, three-parameter mutation with full schema coverage and annotations covering safety, the description is nearly sufficient. The only gap is the absence of any statement about effect of the change or the session ID requirement, which the schema partially covers.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions on all three parameters, so the schema already carries the parameter documentation. The description's note that status_ind comes from the test framework lightly reinforces the enum but adds no syntax or format detail beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (update) and resource (test session) plus the two mutable fields (name, pass/fail status), which is enough to separate it from deleteSession and stopSession. It does not explicitly name siblings or explain how it differs from the tm_update_* tools, so it stops short of a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The clause 'from your test framework' implies the scenario (reporting an external test result) but there is no explicit when-to-use, when-not-to-use, or named alternative. Usage is inferable rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
upload_appUpload App to LambdaTestBInspect
Uploads an app to LambdaTest for mobile testing and displays upload status.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Name for the uploaded app | |
| appUrl | No | URL of the app to upload. Provide this OR appFilePath (exactly one is required). | |
| customId | No | Custom ID for the app (optional) | |
| visibility | No | Visibility setting (optional, default: individual) | |
| appFilePath | No | Absolute path to the app file (APK, IPA, etc.). Provide this OR appUrl (exactly one is required). |
Output Schema
| Name | Required | Description |
|---|---|---|
| appId | No | The app ID |
| error | No | Error message if upload failed |
| appUrl | No | The URL of the uploaded app to use in capabilities |
| message | Yes | Status message |
| success | Yes | Whether the upload was successful |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, and openWorldHint=false, so the agent knows this is a non-destructive write. The description adds that upload status is surfaced, but says nothing about processing latency, async behavior, or whether the uploaded app persists until deleted (see deleteUploadedApps). Adequate but thin against the annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single compact sentence with the verb front-loaded and no filler. It is appropriately sized, though it is too terse to carry any routing or behavioral detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
An output schema exists, so return values need not be described, and the schema fully documents the five parameters including the enum and the either/or upload source. For a straightforward non-destructive upload tool this is complete, though the sibling ambiguity is a missed opportunity rather than a gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the schema itself documents the appUrl/appFilePath mutual exclusivity, the enum for visibility, and which fields are optional. The description contributes no additional parameter semantics beyond the schema, so the baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource (uploads an app to LambdaTest for mobile testing) and adds the side effect of displaying upload status. However, it does not differentiate itself from the very similar sibling uploadAppVirtualDevice, leaving the agent to guess which upload path applies.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use guidance, no prerequisites, and no mention of alternatives. With siblings like uploadAppVirtualDevice, uploadUserFile, and uploadPrerunFile, the agent gets no signal about which upload tool is appropriate for mobile app testing versus virtual devices or generic files.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
uploadAppVirtualDeviceUpload App for Virtual DeviceCInspect
Upload an app (APK for emulator, IPA for simulator) to LambdaTest for virtual device (emulator/simulator) testing.
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | Name for the uploaded app | |
| appUrl | No | Public URL of the app to upload (alternative to filePath) | |
| storage | No | Storage method (default: url when appUrl provided) | |
| filePath | No | Absolute local path to the app file (APK or IPA) | |
| custom_id | No | Custom identifier for the app | |
| visibility | No | Access level (default: individual) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds nothing beyond that: it does not mention authentication requirements, that uploads are asynchronous and require a follow-up processing check, size limits, or what happens to an existing app with the same name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single well-formed sentence that front-loads the verb and resource with no padding. It is appropriately sized, though it is thin enough that the brevity reflects missing detail rather than disciplined editing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries return-value burden and says nothing about the upload response or the resulting app ID. Missing the async-processing handoff to checkAppProcessingStatusVirtualDevice, a near-certain next step in this workflow, is the main gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all six parameters are already documented, including the filePath/appUrl alternative and the two enums. The description only reinforces the accepted file types (APK/IPA), which the schema already encodes. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (Upload) and resource (an app, APK/IPA) with the destination scope (LambdaTest virtual device testing). It distinguishes itself from the many other upload* siblings by naming the emulator/simulator target, though it never explicitly contrasts with upload_app or uploadUserFile.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance. The agent gets no signal about preferring this over upload_app (physical devices), nor about how filePath vs appUrl selection relates to the 'storage' enum. Only the schema hints at the url/filePath alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
24 tool updates
- Changed
automationQuery13 fields changed- changed
Input schema / properties / fromdate / descriptionPrevious value: -"Filter builds from this date (YYYY-MM-DD)"New value: +"Filter builds from this date (YYYY-MM-DD, a real calendar date)" - changed
Input schema / properties / limit / descriptionPrevious value: -"Number of results to return (pagination)"New value: +"Number of results to return, 1 to 100" - added
Input schema / properties / limit / maximumAdded value: +100 - added
Input schema / properties / limit / minimumAdded value: +1 - changed
Input schema / properties / limit / typePrevious value: -"number"New value: +"integer" - changed
Input schema / properties / offset / descriptionPrevious value: -"Offset for pagination"New value: +"Offset for pagination; pass the next offset from a previous response" - added
Input schema / properties / offset / minimumAdded value: +0 - changed
Input schema / properties / offset / typePrevious value: -"number"New value: +"integer" - changed
Input schema / properties / pageNumber / descriptionPrevious value: -"Page number for command log pagination"New value: +"Page number for command log pagination, from 1" - added
Input schema / properties / pageNumber / minimumAdded value: +1 - changed
Input schema / properties / pageNumber / typePrevious value: -"number"New value: +"integer" - changed
Input schema / properties / sessionId / descriptionPrevious value: -"Session ID — required for app platform log retrieval"New value: +"Session ID: required for session get and for app platform log retrieval. Session get also accepts the test ID instead." - changed
Input schema / properties / todate / descriptionPrevious value: -"Filter builds to this date (YYYY-MM-DD)"New value: +"Filter builds to this date (YYYY-MM-DD, not before fromdate)"
- Changed
deleteUploadedApps3 fields changed- changed
Input schema / properties / appIds / descriptionPrevious value: -"Array of app IDs to delete"New value: +"Array of app IDs to delete (at least one)" - added
Input schema / properties / appIds / items / minLengthAdded value: +1 - added
Input schema / properties / appIds / minItemsAdded value: +1
- Changed
getAccessibilityReport1 field changed- added
Input schema / properties / include_rulesAdded value: +{ + "description": "Return the full rule catalog (about 11 KB) instead of the default ruleCatalog summary of counts per category and disabled rule ids.", + "type": "boolean" +}
- Changed
getAnalyticsData1 field changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
- Changed
getAnalyticsErrors1 field changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
- Changed
getFlakyTests1 field changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
- Changed
getLiveSessionLogs1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"The active session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
- Changed
getSessionScreenshots1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
- Changed
getSessionV2CommandLogs1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
- Changed
getSessionV2SeleniumLogs1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
- Changed
getSessionVideo1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
- Changed
getTestTrends1 field changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
- Changed
listDevicesForTesting4 fields changed- added
Input schema / properties / deviceNameAdded value: +{ + "description": "Device name to match, case-insensitively as a substring, e.g. \"Galaxy S24\", \"iPhone 15\", \"Pixel\"", + "type": "string" +} - added
Input schema / properties / limitAdded value: +{ + "description": "Devices per page, 1 to 200 (default 50)", + "maximum": 200, + "minimum": 1, + "type": "integer" +} - added
Input schema / properties / offsetAdded value: +{ + "description": "Devices to skip (default 0)", + "minimum": 0, + "type": "integer" +} - added
Input schema / properties / platformVersionAdded value: +{ + "description": "OS version to match, e.g. \"14\" (matches 14 and 14.x)", + "type": "string" +}
- Changed
listHyperExecuteJobs8 fields changed- changed
Input schema / properties / limit / descriptionPrevious value: -"Maximum number of jobs to return"New value: +"Maximum number of jobs to return, 1 to 200 (default 10)" - added
Input schema / properties / limit / maximumAdded value: +200 - added
Input schema / properties / limit / minimumAdded value: +1 - changed
Input schema / properties / limit / typePrevious value: -"number"New value: +"integer" - changed
Input schema / properties / offset / descriptionPrevious value: -"Pagination offset"New value: +"Number of jobs to skip, after the status filter (default 0)" - added
Input schema / properties / offset / minimumAdded value: +0 - changed
Input schema / properties / offset / typePrevious value: -"number"New value: +"integer" - changed
Input schema / properties / status / descriptionPrevious value: -"Filter by job status (e.g., completed, failed, running)"New value: +"Only jobs with this status (e.g. completed, failed, running, skipped), among the 200 most recent"
- Changed
listMobileBuilds3 fields changed- added
Input schema / properties / limitAdded value: +{ + "description": "Rows per page, 1 to 100 (default 10)", + "maximum": 100, + "minimum": 1, + "type": "integer" +} - added
Input schema / properties / offsetAdded value: +{ + "description": "Rows to skip (default 0)", + "minimum": 0, + "type": "integer" +} - added
Input schema / properties / statusAdded value: +{ + "description": "Only builds with this status, e.g. completed, running, passed, failed", + "type": "string" +}
- Changed
listMobileSessions4 fields changed- added
Input schema / properties / buildIdAdded value: +{ + "description": "Only sessions of this build (build_id from listMobileBuilds)", + "type": "string" +} - added
Input schema / properties / limitAdded value: +{ + "description": "Rows per page, 1 to 100 (default 10)", + "maximum": 100, + "minimum": 1, + "type": "integer" +} - added
Input schema / properties / offsetAdded value: +{ + "description": "Rows to skip (default 0)", + "minimum": 0, + "type": "integer" +} - added
Input schema / properties / statusAdded value: +{ + "description": "Only sessions with this status, e.g. passed, failed, completed, error", + "type": "string" +}
- Changed
listUploadedApps2 fields changed- changed
Input schema / properties / level / descriptionPrevious value: -"Scope: user (default) or team"New value: +"Scope: user (default, apps you uploaded) or team (apps across your organization; sent to the API as level=org)" - added
Input schema / properties / level / enumAdded value: +[ + "user", + "team" +]
- Changed
smartui_build_query1 field changed- changed
Input schema / properties / build_name / descriptionPrevious value: -"Build name — alternative to build_id for get/gate"New value: +"Build name. get/gate: exact name, an alternative to build_id. list: case-insensitive substring filter, applied client-side to the newest 200 builds (upstream has no name filter)."
- Changed
smartui_comparison_analyze1 field changed- changed
Input schema / properties / include_assets / descriptionPrevious value: -"urls (default) returns asset URLs only. inline additionally attaches the screenshots themselves as image blocks — compared only for lens=pixel, baseline+current+compared for consolidated, none for lens=dom. Use inline when you cannot fetch URLs."New value: +"urls (default) returns asset URLs only. inline additionally attaches the screenshots themselves as image blocks — compared only for lens=pixel, baseline+current+compared for consolidated, none for lens=dom. Images over 1.5 MB (or past 2.25 MB in total) are skipped and listed with the reason; their URLs stay in the result. Use inline when you cannot fetch URLs."
- Changed
smartui_screenshot_upload1 field changed- changed
Input schema / properties / screenshot_name / descriptionPrevious value: -"Name this screenshot (binds it to that baseline) instead of using the file/URL name. Only valid when uploading exactly one image."New value: +"Name this screenshot (binds it to that baseline) instead of using the file/URL name. Sent exactly as given, with no extension added, so pass the baseline name as it appears in SmartUI (SDK and CLI captures have no extension). Only valid when uploading exactly one image."
- Changed
tm_create_test_case1 field changed- changed
Input schema / properties / test_steps / items / properties / outcome / descriptionPrevious value: -"Expected result"New value: +"Expected result, shown as the step Outcome in the UI. Use this key: any other step key (such as expected_result) is rejected."
- Changed
tm_get_test_cases1 field changed- changed
Input schema / properties / priority / descriptionPrevious value: -"Filter by priority (e.g. Lowest, Low, Medium, High, Highest)"New value: +"Filter by priority (e.g. Lowest, Low, Medium, High, Highest). A value outside the project's priority list is rejected with the valid values."
- Changed
tm_list_configurations1 field changed- removed
Input schema / properties / project_idRemoved value: -{ - "description": "Optionally scope to a project", - "type": "string" -}
- Changed
tm_update_test_case1 field changed- changed
Input schema / properties / test_steps / items / properties / outcome / descriptionPrevious value: -"Expected result"New value: +"Expected result, shown as the step Outcome in the UI. Use this key: any other step key (such as expected_result) is rejected."
23 tool updates
- Removed
checkAppProcessingStatusVirtualDevice - Removed
deleteHyperExecuteJob - Changed
deletePrerunFile2 fields changed- changed
Input schema / properties / filePath / descriptionPrevious value: -"Stored LambdaTest file path to delete"New value: +"Stored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>)" - changed
Input schema / properties / file_path / descriptionPrevious value: -"Stored LambdaTest file path to delete"New value: +"Stored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>)"
- Changed
deleteUserFile2 fields changed- changed
Input schema / properties / filePath / descriptionPrevious value: -"Stored LambdaTest file path to delete"New value: +"Key of the stored user file, as uploadUserFile returns it (usually the file name)" - changed
Input schema / properties / file_path / descriptionPrevious value: -"Stored LambdaTest file path to delete"New value: +"Key of the stored user file, as uploadUserFile returns it (usually the file name)"
- Changed
generateHyperExecuteYAML20 fields changed- changed
Input schema / properties / appTesting / descriptionPrevious value: -"Mobile app testing configuration. ONLY for mobile testing (XCUI/Espresso/Appium). DO NOT include this for web automation tests (Selenium, Playwright, Cypress, etc.)."New value: +"Mobile app testing configuration (XCUI, Espresso or Appium). Web automation (Selenium, Playwright, Cypress, etc.) leaves it out." - added
Input schema / properties / appTesting / properties / deviceLog / descriptionAdded value: +"Capture device logs" - added
Input schema / properties / appTesting / properties / networkLog / descriptionAdded value: +"Capture network logs" - added
Input schema / properties / appTesting / properties / video / descriptionAdded value: +"Record a video of each test" - added
Input schema / properties / artifacts / properties / mergeArtifacts / descriptionAdded value: +"Merge the artifacts from all parallel runners into one set" - added
Input schema / properties / artifacts / properties / uploadPaths / descriptionAdded value: +"Paths on the runner to upload as job artifacts, e.g. reports/ or screenshots/" - added
Input schema / properties / execution / properties / autosplit / descriptionAdded value: +"Spread the discovered tests across the parallel runners automatically" - added
Input schema / properties / execution / properties / concurrency / descriptionAdded value: +"Number of tests run in parallel" - added
Input schema / properties / execution / properties / maxRetries / descriptionAdded value: +"Retries per failed test when retryOnFailure is on" - added
Input schema / properties / execution / properties / os / descriptionAdded value: +"Runner operating system" - added
Input schema / properties / execution / properties / retryOnFailure / descriptionAdded value: +"Retry failed tests" - added
Input schema / properties / setup / properties / runtime / properties / language / descriptionAdded value: +"Runtime to install on the runner, e.g. node, python, java" - added
Input schema / properties / setup / properties / runtime / properties / version / descriptionAdded value: +"Runtime version, e.g. 20 or 3.11" - changed
Input schema / properties / sourcePayload / descriptionPrevious value: -"Git integration - ONLY include this if the user explicitly requests fetching code from a remote Git repository. Do NOT include this when code is already present locally (which is the default case). Most users run tests on local code and do not need sourcePayload."New value: +"Git integration: fetches the test code from a remote Git repository. Only needed when the code is not already present locally; local code (the usual case) leaves it out." - changed
Input schema / properties / sourcePayload / properties / accessToken / descriptionPrevious value: -"Use env var reference: ${GIT_TOKEN}"New value: +"Reference to an environment variable that holds the Git token, e.g. ${GIT_TOKEN}; never the token itself" - added
Input schema / properties / sourcePayload / properties / gitProvider / descriptionAdded value: +"Git host of the repository" - added
Input schema / properties / sourcePayload / properties / platform / descriptionAdded value: +"Source platform; git is the only supported value" - added
Input schema / properties / testing / properties / testDiscovery / descriptionAdded value: +"How HyperExecute finds the tests to run" - added
Input schema / properties / testing / properties / testDiscovery / properties / mode / descriptionAdded value: +"dynamic discovers tests on the runner at run time; static uses the list found before the run" - added
Input schema / properties / testing / properties / testDiscovery / properties / type / descriptionAdded value: +"How tests are discovered: raw runs the command as given, automatic lets HyperExecute detect them"
- Removed
getAccessibilityScanIssues - Removed
getAccessibilityScanReport - Changed
getAutomationTestCommandLogs1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"Required for app platform: the session ID of the app test"New value: +"Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)"
- Changed
getAutomationTestNetworkLogs1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"Required for app platform: the session ID of the app test"New value: +"Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)"
- Changed
getAutomationTestSeleniumLogs1 field changed- changed
Input schema / properties / sessionId / descriptionPrevious value: -"Required for app platform: the session ID of the app test"New value: +"Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)"
- Removed
getPerformanceTestResult - Removed
getScreenshotTestResult - Removed
postSessionAssertions - Changed
postSessionExceptions4 fields changed- changed
Input schema / properties / data / additionalPropertiesPrevious value: -trueNew value: +false - changed
Input schema / properties / data / descriptionPrevious value: -"Exception payload to upload"New value: +"The exceptions to attach: {\"exception\": [\"first message\", \"second message\"]}" - added
Input schema / properties / data / propertiesAdded value: +{ + "exception": { + "description": "Exception messages, one string each", + "items": { + "type": "string" + }, + "minItems": 1, + "type": "array" + } +} - added
Input schema / properties / data / requiredAdded value: +[ + "exception" +]
- Changed
postTestExceptions4 fields changed- changed
Input schema / properties / data / additionalPropertiesPrevious value: -trueNew value: +false - changed
Input schema / properties / data / descriptionPrevious value: -"Exception payload to upload"New value: +"The exceptions to attach: {\"exception\": [\"first message\", \"second message\"]}" - added
Input schema / properties / data / propertiesAdded value: +{ + "exception": { + "description": "Exception messages, one string each", + "items": { + "type": "string" + }, + "minItems": 1, + "type": "array" + } +} - added
Input schema / properties / data / requiredAdded value: +[ + "exception" +]
- Removed
stopScreenshotTest - Changed
tm_create_milestone1 field changed- changed
Input schema / properties / owner / descriptionPrevious value: -"Milestone Owner — an organization member (email, username, or user id). ASK the user who owns it if unspecified."New value: +"Milestone owner: an organization member (email, username, or user id). Omit for no owner."
- Changed
tm_create_test_run1 field changed- changed
Input schema / properties / assignee / descriptionPrevious value: -"Assign every created instance to this organization member — email, username, or user id. If the user has not said who to assign, ASK them (or confirm no assignee) before creating; never guess."New value: +"Assign every created instance to this organization member: email, username, or user id. Omit to leave the instances unassigned."
- Changed
tm_generate_test_cases1 field changed- changed
Input schema / properties / files / descriptionPrevious value: -"Optional reference files to ground the generation (requirement docs, specs, data sheets, screenshots). Supported: .pdf, .docx, .txt, .csv, .xlsx, .json, .xml, .jpg, .jpeg, .png, .gif, .bmp, .webp — max 5 files, 10 MB each. Read the file yourself and base64-encode its raw bytes (shell: base64 -i <file>). Prefer small files (up to a few hundred KB) — base64 content passes through your context, so upload larger files via the Test Manager UI instead. Only used when starting a new generation."New value: +"Optional reference files to ground the generation (requirement docs, specs, data sheets, screenshots). Each item carries the file name and its raw bytes base64-encoded in content_base64. Supported: .pdf, .docx, .txt, .csv, .xlsx, .json, .xml, .jpg, .jpeg, .png, .gif, .bmp, .webp; max 5 files, 10 MB each. Large files are better uploaded in the Test Manager UI, since the base64 content travels in the request. Only used when starting a new generation."
- Changed
tm_record_test_results1 field changed- changed
Input schema / properties / results / items / properties / assignee / descriptionPrevious value: -"Assign this instance to an organization member (email, username, or user id; \"none\" clears). ASK the user if unspecified."New value: +"Assign this instance to an organization member (email, username, or user id; \"none\" clears). Omit to keep the current assignee."
- Changed
tm_update_test_run1 field changed- changed
Input schema / properties / assignee / descriptionPrevious value: -"Assign the run's instances to this organization member — email, username, or user id (\"none\" clears). Scope with apply_to_test_case_ids. ASK the user who to assign if unspecified; never guess."New value: +"Assign the run's instances to this organization member: email, username, or user id (\"none\" clears). Scope with apply_to_test_case_ids. Omit to keep the current assignee."
- Removed
uploadPrerunFile - Removed
uploadUserFile
16 tool updates
- Changed
automationQuery4 fields changed- changed
Input schema / properties / action / descriptionPrevious value: -"The action to perform on the resource"New value: +"The read to perform: list, or get one" - changed
Input schema / properties / action / enumPrevious value: -[ - "list", - "get", - "delete", - "update", - "stop" -]New value: +[ + "list", + "get" +] - changed
Input schema / properties / buildId / descriptionPrevious value: -"Build ID — required for build get/delete/update/stop and optional for session list"New value: +"Build ID — required for build get, optional filter for session list" - removed
Input schema / properties / nameRemoved value: -{ - "description": "New name for build update", - "type": "string" -}
- Added
deleteBuild - Changed
getAnalyticsData4 fields changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date (ISO 8601 format, e.g. 2024-01-01)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate." - changed
Input schema / properties / groupBy / descriptionPrevious value: -"Dimension to group results by (e.g., browser, os, status)"New value: +"Dimension to group by. Defaults to status." - added
Input schema / properties / groupBy / enumAdded value: +[ + "status", + "browser", + "os", + "build", + "device", + "day" +] - changed
Input schema / properties / toDate / descriptionPrevious value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
- Changed
getAnalyticsErrors3 fields changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date (ISO 8601 format)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate." - changed
Input schema / properties / limit / descriptionPrevious value: -"Maximum number of results to return"New value: +"Maximum failing tests to list, 1 to 200. Defaults to 20." - changed
Input schema / properties / toDate / descriptionPrevious value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
- Changed
getFlakyTests3 fields changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date (ISO 8601 format)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate." - changed
Input schema / properties / limit / descriptionPrevious value: -"Maximum number of results to return"New value: +"Maximum flaky tests to return, 1 to 200. Defaults to 50." - changed
Input schema / properties / toDate / descriptionPrevious value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
- Changed
getHyperExecuteJobInfo2 fields changed- removed
Input schema / properties / lt_access_keyRemoved value: -{ - "description": "LambdaTest access key (optional)", - "type": "string" -} - removed
Input schema / properties / lt_usernameRemoved value: -{ - "description": "LambdaTest username (optional)", - "type": "string" -}
- Changed
getHyperExecuteJobSessions2 fields changed- removed
Input schema / properties / lt_access_keyRemoved value: -{ - "description": "LambdaTest access key (optional)", - "type": "string" -} - removed
Input schema / properties / lt_usernameRemoved value: -{ - "description": "LambdaTest username (optional)", - "type": "string" -}
- Changed
getSupportedPlatforms3 fields changed- added
Input schema / properties / browserAdded value: +{ + "description": "Browser to expand on desktop, e.g. \"chrome\", \"firefox\", \"edge\", \"safari\"", + "type": "string" +} - added
Input schema / properties / osAdded value: +{ + "description": "OS name to expand, matched case-insensitively as a substring, e.g. \"Windows 11\", \"Sonoma\", \"android\"", + "type": "string" +} - added
Input schema / properties / typeAdded value: +{ + "description": "Only desktop OSes or only mobile platforms", + "enum": [ + "desktop", + "mobile" + ], + "type": "string" +}
- Changed
getTestTrends2 fields changed- changed
Input schema / properties / fromDate / descriptionPrevious value: -"Start date (ISO 8601 format)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate." - changed
Input schema / properties / toDate / descriptionPrevious value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
- Changed
smartui_baseline_manage2 fields changed- changed
Input schema / properties / action / descriptionPrevious value: -"get_settings reads baseline config; merge_branch/merge_build merge baselines"New value: +"merge_branch merges one branch baseline into another; merge_build merges one build into another" - changed
Input schema / properties / action / enumPrevious value: -[ - "get_settings", - "merge_branch", - "merge_build" -]New value: +[ + "merge_branch", + "merge_build" +]
- Changed
smartui_screenshot_upload1 field changed- removed
Input schema / properties / project_tokenRemoved value: -{ - "description": "Explicit projectToken (alternative to project_id)", - "type": "string" -}
- Added
stopBuild - Changed
tm_list_configurations1 field changed- changed
Input schema / properties / limit / descriptionPrevious value: -"Max results per page"New value: +"Max results per page (default 10)"
- Changed
tm_list_projects1 field changed- changed
Input schema / properties / limit / descriptionPrevious value: -"Max results per page"New value: +"Max results per page (default 25)"
- Changed
triggerSuiteRun2 fields changed- removed
Input schema / properties / accessKeyRemoved value: -{ - "description": "Access key for Basic Auth (optional, uses authenticated credentials if not provided)", - "type": "string" -} - removed
Input schema / properties / usernameRemoved value: -{ - "description": "Username for Basic Auth (optional, uses authenticated user if not provided)", - "type": "string" -}
- Added
updateBuild
95 tool updates
- First observed
analyzeAppViaTunnel - First observed
analyzeHyperExecuteJob - First observed
answerHyperExecuteQuery - First observed
automationQuery - First observed
buildLocalAppForAnalysis - First observed
checkAppProcessingStatus - First observed
checkAppProcessingStatusVirtualDevice - First observed
deleteHyperExecuteJob - First observed
deletePrerunFile - First observed
deleteSession - First observed
deleteUploadedApps - First observed
deleteUserFile - First observed
downloadJobArtifact - First observed
generateHyperExecuteYAML - First observed
getAccessibilityReport - First observed
getAccessibilityScanIssues - First observed
getAccessibilityScanReport - First observed
getAnalyticsData - First observed
getAnalyticsErrors - First observed
getAutomationTestBrowserConsoleLogs - First observed
getAutomationTestCommandLogs - First observed
getAutomationTestDetails - First observed
getAutomationTestNetworkLogs - First observed
getAutomationTestSeleniumLogs - First observed
getFlakyTests - First observed
getHyperExecuteJobArtifacts - First observed
getHyperExecuteJobInfo - First observed
getHyperExecuteJobLiveStream - First observed
getHyperExecuteJobSessions - First observed
getHyperExecuteStage - First observed
getHyperExecuteStageLogs - First observed
getHyperExecuteStageTasks - First observed
getHyperExecuteTask - First observed
getHyperExecuteTaskLogs - First observed
getJobArtifactDetail - First observed
getLiveSessionLogs - First observed
getMobileCapabilityGenerator - First observed
getMobileConcurrency - First observed
getOrgConcurrency - First observed
getPerformanceTestResult - First observed
getScreenshotTestResult - First observed
getSessionScreenshots - First observed
getSessionV2CommandLogs - First observed
getSessionV2SeleniumLogs - First observed
getSessionVideo - First observed
getSuiteOverview - First observed
getSupportedPlatforms - First observed
getTestTrends - First observed
listDevicesForTesting - First observed
listHyperExecuteJobs - First observed
listMobileBuilds - First observed
listMobileSessions - First observed
listTunnels - First observed
listUploadedApps - First observed
postSessionAssertions - First observed
postSessionExceptions - First observed
postTestExceptions - First observed
smartui_baseline_manage - First observed
smartui_build_query - First observed
smartui_comparison_analyze - First observed
smartui_comparison_query - First observed
smartui_project_query - First observed
smartui_screenshot_review - First observed
smartui_screenshot_upload - First observed
stopScreenshotTest - First observed
stopSession - First observed
stopTunnel - First observed
tm_create_folder - First observed
tm_create_milestone - First observed
tm_create_project - First observed
tm_create_test_case - First observed
tm_create_test_run - First observed
tm_generate_test_cases - First observed
tm_get_coverage_summary - First observed
tm_get_milestone - First observed
tm_get_test_cases - First observed
tm_get_test_runs - First observed
tm_link_ado_issue - First observed
tm_link_jira_issue - First observed
tm_list_configurations - First observed
tm_list_folders - First observed
tm_list_modules - First observed
tm_list_projects - First observed
tm_record_test_results - First observed
tm_update_folder - First observed
tm_update_milestone - First observed
tm_update_project - First observed
tm_update_test_case - First observed
tm_update_test_run - First observed
triggerSuiteRun - First observed
updateSession - First observed
upload_app - First observed
uploadAppVirtualDevice - First observed
uploadPrerunFile - First observed
uploadUserFile
Related MCP Connectors
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Official MCP server for Qase — manage test cases, runs, suites, defects via AI tools.
Approved test intent, reviewed Playwright automation and run evidence, inside your editor.
MCP server for Mint — AI-powered QA that runs your app in a real browser on every PR.
Related MCP Servers
- FlicenseNot gradedqualityCmaintenanceExposes AI agents as MCP tools to fetch Jira stories, generate BRDs and Playwright test scripts, run tests, and auto-heal broken locators. Integrates with Claude Desktop and Cursor IDE for enterprise-grade test automation.-
- FlicenseNot gradedqualityCmaintenanceMCP server for the Argus E2E Testing Agent, enabling AI coding assistants to perform browser testing, quality intelligence, CI/CD integration, discovery, time travel debugging, visual AI testing, correlation analytics, and API testing directly from the IDE.-
- AlicenseAqualityAmaintenanceMCP server for Allure TestOps — lets an LLM agent (Claude Code, Cursor, Windsurf, etc.) inspect test quality without leaving the chat.11138 PyPI3MIT
- AlicenseNot gradedqualityDmaintenanceA universal AI-powered testing server built on the Model Context Protocol (MCP). Allows AI agents to inspect, execute, test, monitor, debug, and report on software projects.3GNU Lesser General Public v2.1 only
Glama MCP Gateway
Add one secure layer between your agents and this server.