Skip to main content
Glama

MCP Server By TestMu AI

Server Details

Run, debug, and triage tests from your IDE using natural language, no dashboard switching, no manual data transfers.

The TestMu AI (formerly LambdaTest) MCP Server is a single remote server exposing four tool suites:

HyperExecute — analyze your project, generate YAML configs and test runner commands, then monitor jobs and sessions. Automation — pull a TestID's details plus command, network, and console logs into one chat for instant root-cause analysis. Includes mobile app upload. SmartUI — explain pixel, layout, DOM, and perceptual changes in a visual regression run, with context-aware React/HTML/CSS fixes. Accessibility — audit any public URL or a local React app against WCAG and get ready-to-apply remediation steps.

Connects over https://mcp.lambdatest.com/mcp using OAuth 2.1 — no API keys in your config. One-click install in Cursor; works with Claude, GitHub Copilot, Cline, and any MCP client.

Tests execute on the TestMu AI cloud: 3,000+ browsers and 10,000+ real devices.

Ownership verified
Status
Healthy
Uptime
24.6% over 66 days
OAuth
Works in Glama
Last Tested
Transport
Streamable HTTP · MCP 2025-03-26
URL

TDQS

B3.1/5.0

Scored across 88 tools

Disambiguation3/5

The 88-tool surface spans many LambdaTest products and includes several overlapping retrieval paths (e.g., getAutomationTest* vs getSessionV2* logs, multiple granular HyperExecute getters vs analyzeHyperExecuteJob). Descriptions do explain boundaries and v1/v2 distinctions, but the sheer number of similar-sounding tools leaves meaningful misselection risk.

Naming Consistency2/5

Most tools use camelCase verb_noun (getBuild, deleteSession, listTunnels), but entire groups use snake_case with prefixes (smartui_*, tm_*) and one tool is upload_app. This mixes camelCase and snake_case conventions across the set, making naming unpredictable despite some group-level consistency.

Tool Count1/5

88 tools is far beyond the typical 3–15 range and even beyond the 25+ heavy threshold. While the server covers a broad enterprise platform, this count is an extreme mismatch for a single MCP tool set and creates a high cognitive load for an agent.

Completeness3/5

Coverage is broad across web/mobile automation, HyperExecute, SmartUI, Test Manager, accessibility, and tunnels, with many create/read/update/delete operations present. However, notable gaps exist: Test Manager deletion is explicitly unavailable for projects, folders, test cases, and milestones; there is no stop/abort for HyperExecute jobs; and tunnel creation is not directly supported.

Available Tools

88 tools
analyzeAppViaTunnelAnalyze App via TunnelB
Read-only
Inspect

Analyzes a locally running app for accessibility via LambdaTest tunnel.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort where the app is running
tunnelIdYesTunnel ID
tunnelNameYesTunnel name

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=false, and destructiveHint=false, so safety is covered. The description adds that the app must be local and tunneled, but says nothing about whether the analysis is synchronous, whether a status poll (e.g. checkAppProcessingStatus) is needed, or what triggers the scan.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single tight sentence with the key scoping qualifier ('locally running', 'via tunnel') front-loaded. Nothing is wasted, though it is arguably too terse to carry the workflow context an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should clarify what the call yields (e.g. a scan/report id) and whether further steps are required. It leaves the completion path implicit despite the presence of related siblings like checkAppProcessingStatus and getAccessibilityScanReport.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and all three parameters (port, tunnelId, tunnelName) are self-explanatory and documented in the schema. The description adds no extra meaning beyond what the schema provides, which is the expected baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Analyzes), the resource (a locally running app), the goal (accessibility), and the transport (via LambdaTest tunnel). It is clear about what it does, though it does not explicitly contrast itself with nearby siblings such as buildLocalAppForAnalysis or getAccessibilityReport.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'locally running app ... via LambdaTest tunnel' implies the preconditions (an app served on a local port reachable through a tunnel), but there is no explicit when-to-use vs when-not guidance or named alternative for already-uploaded apps.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

analyzeHyperExecuteJobAnalyze HyperExecute JobA
Read-only
Inspect

Perform a full analysis of a HyperExecute job: fetches job info, sessions, traverses all tasks → scenario stages → console logs, and returns a structured report with test results, per-scenario logs, and pass/fail statistics. Use this for questions like "which tests failed and why" or "show me the console logs for job X".

ParametersJSON Schema
NameRequiredDescriptionDefault
jobIdYesThe HyperExecute job ID to analyze

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is lower. The description still adds useful behavioral context: it discloses that the tool makes a deep recursive traversal of all tasks and stages and returns an aggregated structured report rather than a single resource, implying heavier cost/latency than the granular siblings.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no waste. The core capability is front-loaded in the first sentence and the second sentence supplies the routing examples, so the reader gets purpose before use cases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by naming what the report contains (test results, per-scenario logs, pass/fail statistics). Combined with annotations covering safety and a fully documented single required parameter, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter with 100% schema description coverage, so the schema already documents jobId fully. The description adds only marginal meaning (that the id is the job to analyze), which is the expected baseline when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (analyze/fetch/traverse) and resource (HyperExecute job) and enumerates exactly what the deep traversal covers: job info, sessions, tasks → scenario stages → console logs, plus the returned report contents. This scope description makes it distinguishable from the granular per-resource siblings like getHyperExecuteJobInfo or getHyperExecuteStageLogs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly gives use cases with two concrete example questions ('which tests failed and why', 'show me the console logs for job X'), which tells the agent when to pick this aggregate tool. It stops short of naming alternatives such as answerHyperExecuteQuery or the individual getter tools, so there is no when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

answerHyperExecuteQueryAnswer HyperExecute QuestionB
Read-only
Inspect

Answers questions about HyperExecute features, configuration, usage, troubleshooting, and best practices by searching through documentation.

ParametersJSON Schema
NameRequiredDescriptionDefault
queryYesQuestion about HyperExecute

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety and closed-domain profile is covered. The description adds that answers are sourced from documentation (not live job state), which is useful, but it says nothing about grounding, citation behavior, or answer completeness.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that spends its words on the tool's purpose and mechanism with no filler. The enumerated topics are slightly list-heavy but each earns its place by sketching the question domain.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and a trivial input schema, the description covers the essentials: what it answers and that it searches documentation. It stops short of describing the answer format, source citations, or limitations, which would help an agent know what to expect back.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter with 100% schema description coverage, so the schema already defines 'query' as a question about HyperExecute. The description adds no syntax, phrasing, or scope detail beyond that, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('answers questions about HyperExecute features...') and names the mechanism ('searching through documentation'). It clearly delineates the topical scope, though it does not explicitly distinguish itself from any named sibling tool.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the topical scope (HyperExecute questions), but there is no explicit when-to-use/when-not guidance or pointer to alternatives such as analyzeHyperExecuteJob for job-specific data rather than general documentation questions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

automationQueryAutomation QueryA
Read-only
Inspect

Read-only query of LambdaTest automation resources: list builds, get one build, list the sessions in a build, fetch test details, and retrieve test logs (command, network, console, selenium, instrumentation). Every combination works on platform "web" or "app": build list or get, session list or get, test get, and logs get with a logType. List results carry a pagination object; trust it, not result_set.total, which also counts older builds the list no longer returns. logType "instrumentation" returns XCUI and Espresso test logs. Renaming, stopping and deleting builds are separate tools: updateBuild, stopBuild and deleteBuild.

ParametersJSON Schema
NameRequiredDescriptionDefault
sortNoSort order for build listing (e.g. "asc", "desc")
limitNoNumber of results to return, 1 to 100
actionYesThe read to perform: list, or get one
offsetNoOffset for pagination; pass the next offset from a previous response
statusNoFilter builds by status (e.g. "running", "passed", "failed")
testIdNoTest ID — required for test get and logs get
todateNoFilter builds to this date (YYYY-MM-DD, not before fromdate)
buildIdNoBuild ID — required for build get, optional filter for session list
logTypeNoLog type — required when resource is "logs"
fromdateNoFilter builds from this date (YYYY-MM-DD, a real calendar date)
platformYesThe platform type: "web" for browser automation, "app" for mobile app automation
resourceYesThe resource type to query
sessionIdNoSession ID: required for session get and for app platform log retrieval. Session get also accepts the test ID instead.
pageNumberNoPage number for command log pagination, from 1

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds meaningful behavioral context beyond annotations: the pagination-object caveat versus result_set.total, and the fact that logType 'instrumentation' returns XCUI and Espresso logs. It omits auth constraints and rate limits, so it is strong but not exhaustive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core purpose, then lists capabilities and caveats in a compact, information-dense paragraph. Every sentence earns its place: the resource enumeration, platform applicability, pagination warning, and sibling routing all add distinct value with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 14 parameters, a multi-action dispatcher design, and no output schema, the description covers the essential call semantics: resource/action combinations, platform scope, logType behavior, and pagination handling. It does not describe return shapes for get/list operations or error behavior, but for a read-only query tool with rich schema coverage this is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter, including required combinations and logType enums. The description reinforces useful relational semantics—every resource/action combination works on web or app, and logs require a logType—but adds little syntax or format detail beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource family: 'Read-only query of LambdaTest automation resources', then enumerates the exact reads it covers (builds, sessions, tests, logs). It distinguishes itself from mutation siblings by explicitly naming updateBuild, stopBuild, and deleteBuild, so an agent can tell it apart without inspecting schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear when-to-use context: read-only automation queries across web and app, with explicit exclusions for build mutations (updateBuild, stopBuild, deleteBuild). However, it does not distinguish itself from specialized log-retrieval siblings like getAutomationTestCommandLogs or getSessionV2CommandLogs, leaving overlap unresolved.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

buildLocalAppForAnalysisBuild and Analyze AppA
Read-only
Inspect

Returns step-by-step instructions to build and serve a local web app and open a LambdaTest tunnel to it, so its accessibility can then be scanned with analyzeAppViaTunnel. It does not build, run or scan anything itself, and never includes the access key.

ParametersJSON Schema
NameRequiredDescriptionDefault
portYesPort to host the app locally
codebasePathYesAbsolute path to the codebase directory
customBuildCommandNoCustom build command (optional, e.g., npm run build)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds genuinely new behavioral context: it performs no build/run/scan side effects itself, and it guarantees the access key is never emitted — a meaningful security disclosure for a tool whose output is instructions containing tunnel setup steps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler; the core purpose is front-loaded and the second sentence adds only the two constraints that matter (no execution, no access key). Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must convey what is returned, and it does — step-by-step build/serve/tunnel instructions. It also covers the relationship to the downstream scan tool and the scope limits, which is everything an agent needs to call and use it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and all three parameters (port, codebasePath, customBuildCommand) carry their own descriptions, so the schema does the heavy lifting. The description adds no parameter-level detail such as port/range constraints or how customBuildCommand interacts with the default build — baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and output ('Returns step-by-step instructions to build and serve a local web app and open a LambdaTest tunnel'), and explicitly disclaims doing the building, running or scanning itself. It also names the sibling tool (analyzeAppViaTunnel) that consumes the result, so it is distinguishable from siblings without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes the workflow ordering explicit: this tool produces instructions so accessibility 'can then be scanned with analyzeAppViaTunnel', which tells the agent where it sits relative to that sibling. It doesn't state when to prefer this over, say, analyzeHyperExecuteJob or direct uploads, but the sequencing context is clear and actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

checkAppProcessingStatusCheck App Processing Status (Real Device)A
Idempotent
Inspect

Check whether an uploaded app has been processed and is ready for features like image injection, network logs, and screenshot unblock on real devices. The LambdaTest API has no read-only status call: this one starts or refreshes the processing for the requested features, so the app's last-updated time changes. An app id that is not among your uploaded apps returns an error.

ParametersJSON Schema
NameRequiredDescriptionDefault
appIdYesThe app ID to check
networkLogsEnabledNoCheck network logs support
imageInjectionEnabledNoCheck image injection support
screenshotUnblockEnabledNoCheck screenshot unblock support

TDQS

A4.1/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds substantial behavior beyond the annotations: the API exposes no read-only status call, so this call starts or refreshes processing, mutating the app's last-updated timestamp, and an unknown app id returns an error. This directly resolves the trap of reading a 'check' tool as a safe read (annotations do correctly declare readOnlyHint=false).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no redundancy, and the mutated-state warning is front-and-center right after the purpose. Every sentence carries distinct information (readiness semantics, mutation side effect, error behavior).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description should ideally hint at what the returned status looks like (e.g., pending/processed/failed and per-feature readiness), which is the one gap for a 'check status' tool. Otherwise it covers purpose, side effects, and error behavior adequately for a 4-param mutation-adjacent tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all four parameters, making 3 the baseline. The description earns above baseline by explaining that the boolean flags correspond to 'the requested features' whose processing is started/refreshed, tying the flags to an observable side effect rather than just 'check X support'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('check whether an uploaded app has been processed') and scopes it to real devices and the features it gates (image injection, network logs, screenshot unblock). It cannot be confused with listUploadedApps or deleteUploadedApps, though it never names a sibling tool to route away from.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The implied usage is clear enough — call it to confirm an uploaded app is ready before relying on injection/network-log/screenshot-unblock features. However, it never states when NOT to call it, names no alternative, and gives no precondition such as 'requires the app to be uploaded first'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteBuildDelete Automation BuildA
DestructiveIdempotent
Inspect

Permanently deletes a LambdaTest web automation build and its test results. Cannot be undone.

ParametersJSON Schema
NameRequiredDescriptionDefault
buildIdYesID of the build to delete

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and idempotentHint=true, so the safety profile is known. The description adds real value beyond that: it discloses cascading deletion of associated test results and irreversibility, which the annotations do not express.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two terse sentences with the destructive scope front-loaded and the irreversibility warning immediately after. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter mutation tool with full annotation coverage and no output schema, the description covers the essential unknowns: permanence and cascade scope. Missing only edge-case context such as permission requirements or behavior when the build does not exist.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Single buildId parameter with 100% schema description coverage, so the baseline is 3. The description adds no format, type, or sourcing detail for buildId beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (permanently deletes) and resource (LambdaTest web automation build plus its test results). This clearly separates it from stopBuild and updateBuild in the sibling list, which act on the same resource with different semantics.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to delete versus stop (stopBuild) or update (updateBuild) a build, and no mention of prerequisites or authorization. The 'Cannot be undone' warning signals severity but is not a usage routing rule.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deletePrerunFileDelete Prerun FileB
Destructive
Inspect

Delete an uploaded prerun file by stored LambdaTest file path.

ParametersJSON Schema
NameRequiredDescriptionDefault
filePathNoStored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>)
file_pathNoStored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>)

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, readOnlyHint=false, and openWorldHint=false, so the safety profile is covered. The description adds only the path-based identity mechanism and does not discuss irreversibility, required permissions, or what happens to a file still referenced by a pre-run/post-run configuration.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler, appropriately sized for a simple delete operation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive tool the description covers the bare minimum: what is deleted and by what key. With no output schema and zero required parameters, it omits return/confirmation behavior and any caveat about the duplicate path parameter, so it is adequate but thin.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. However, the schema exposes two near-identical parameters (filePath and file_path) and the description does not clarify which one is authoritative or that the value must originate from uploadPrerunFile, leaving a real ambiguity unaddressed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Delete') and resource ('an uploaded prerun file') plus the identifier mechanism ('stored LambdaTest file path'). It is clear what the tool does, but it does not distinguish itself from similar siblings such as deleteUserFile or deleteUploadedApps.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance, no preconditions, and no mention of alternatives despite several delete-oriented siblings (deleteUserFile, deleteUploadedApps, deleteBuild, deleteSession) that an agent must choose between. The agent is left to infer the distinction entirely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteSessionDelete SessionA
Destructive
Inspect

Permanently delete a specific automation test session from LambdaTest by session ID.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe session ID to delete

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, readOnlyHint=false and openWorldHint=false, so the destructive nature is known. 'Permanently' reinforces irreversibility, which is a small addition, but nothing is said about permissions, side effects, or whether the parent job is affected.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence naming verb, resource, qualifier and identifier. Zero waste, no repetition of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive tool whose annotations already cover the safety profile and with no output schema, the description is essentially sufficient. Only the sibling-choosing guidance is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single parameter has its own description ('The session ID to delete'), so the schema carries the semantics. The description adds no format or sourcing detail beyond it; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (permanently delete), resource (automation test session), and scope (by session ID) on LambdaTest. It does not explicitly name the near-sibling stopSession, but 'permanently' implicitly distinguishes it from a mere stop.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: delete a session you no longer need. However, it never states when to prefer stopSession over deleteSession, or any prerequisites (ownership, active vs completed session). No explicit when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteUploadedAppsDelete Uploaded AppsA
Destructive
Inspect

Delete one or more uploaded apps from LambdaTest by their app IDs.

ParametersJSON Schema
NameRequiredDescriptionDefault
appIdsYesArray of app IDs to delete (at least one)

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, readOnlyHint=false, and openWorldHint=false, so the safety profile is covered externally. The description confirms deletion but adds no depth beyond that — no mention of irreversibility, whether apps in active use are blocked, or any confirmation requirement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler; the verb, resource, and key are all in the first clause.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter delete operation whose safety profile is carried by annotations, the description is largely sufficient and no output schema is needed. It stops short of noting whether deletion is permanent or has side effects on linked test runs, which would round it out.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with the single appIds parameter fully documented as an array with minItems 1. The description's 'by their app IDs' and 'one or more' merely restate the schema, adding no format or constraint detail beyond it. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (delete), resource (uploaded apps), scope (one or more), and the key used (app IDs). An agent can distinguish this from sibling deletion tools like deleteBuild, deleteSession, and deleteUserFile without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage through 'delete one or more uploaded apps' but never states when to use this over listUploadedApps or the upload tools, nor any preconditions. No when/when-not guidance or named alternatives are present.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

deleteUserFileDelete User FileB
Destructive
Inspect

Delete an uploaded user file by stored LambdaTest file path.

ParametersJSON Schema
NameRequiredDescriptionDefault
filePathNoKey of the stored user file, as uploadUserFile returns it (usually the file name)
file_pathNoKey of the stored user file, as uploadUserFile returns it (usually the file name)

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and readOnlyHint=false, so the agent knows this is an irreversible write; the description correctly does not contradict that. However it adds almost nothing on top of the annotations: no statement that deletion is permanent, no auth/permission requirement, no behavior when the path does not exist. With annotations carrying the safety profile, this is a minimum-viable 3.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with the verb and the identifying key in the first few words, and zero filler or restatement of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive delete with no output schema, the description covers the essential purpose but leaves the notable ambiguity in the schema unresolved: filePath and file_path are both optional and identically described, and the description does not say which one is canonical or whether either is required. It is adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both parameters are already documented as 'Key of the stored user file, as uploadUserFile returns it'. The description repeats the same idea ('by stored LambdaTest file path') without adding syntax, format, or precedence guidance, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb (Delete) and resource (an uploaded user file) qualified by an identifier source (stored LambdaTest file path), which is enough to separate it from deleteBuild, deleteSession, and deletePrerunFile. It does not explicitly name those siblings or state what distinguishes a 'user file' from a 'prerun file', so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no mention of the neighboring delete tools (deletePrerunFile, deleteUploadedApps) that an agent must choose between. The only implicit cue is the word 'uploaded', which does not constitute routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

downloadJobArtifactDownload Job ArtifactA
Read-only
Inspect

Get an authenticated download URL (plus size/status metadata) for a specific HyperExecute job artifact. Requires artefactName (from getHyperExecuteJobArtifacts); optionally scope to a task with taskId. Returns a URL to fetch with your LambdaTest credentials rather than the raw file bytes.

ParametersJSON Schema
NameRequiredDescriptionDefault
jobIdYesThe HyperExecute job ID whose artifact to download
taskIdNoOptional: scope the artifact to a specific task ID
artefactNameYesName of the artifact to download (as returned by getHyperExecuteJobArtifacts)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false). The description adds genuinely useful behavior beyond them: it returns a URL rather than bytes, and that URL must be fetched with LambdaTest credentials, which is critical for correct downstream use.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences: what it returns, the prerequisites, and the key behavioral caveat. Nothing is redundant and the most important fact (URL, not bytes) is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden and does so by naming the URL plus size/status metadata. Auth requirements, the required input source, and optional scoping are all covered for a simple three-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters are already documented, including that artefactName comes from getHyperExecuteJobArtifacts. The description restates that origin but adds no new format, validation, or scoping semantics. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Get an authenticated download URL ... for a specific HyperExecute job artifact') and clarifies the return contract ('URL to fetch ... rather than the raw file bytes'). This cleanly separates it from getHyperExecuteJobArtifacts (which lists names) and getJobArtifactDetail.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It names the prerequisite source for artefactName ('from getHyperExecuteJobArtifacts') and explains when to use the optional taskId scope. It gives clear context for use but does not state explicit exclusions or contrast with the closest sibling beyond that prerequisite mention.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

generateHyperExecuteYAMLGenerate HyperExecute YAMLA
Read-only
Inspect

Generates a HyperExecute YAML configuration for running tests on LambdaTest's HyperExecute platform.

REQUIRED fields: project (language, framework) and testing (testDiscovery.command, testRunnerCommand) All other fields are OPTIONAL.

Supports multiple languages and frameworks:

  • JavaScript/TypeScript: Playwright, Cypress, WebDriverIO, Puppeteer, Taiko

  • Python: Pytest, Behave, Robot Framework

  • Java: TestNG, JUnit, Cucumber

  • Ruby: RSpec, Capybara

  • C#: NUnit, xUnit, MSTest

  • Mobile (ONLY use appTesting for these): XCUI (iOS), Espresso (Android), Appium

Features:

  • Smart framework defaults for common configurations

  • Input validation with security checks

  • Caching and artifact upload configuration

  • Git integration via sourcePayload

  • Mobile app testing support (via optional appTesting field)

The appTesting field applies to mobile app testing only; web automation (Selenium, Playwright, Cypress, etc.) leaves it out.

ParametersJSON Schema
NameRequiredDescriptionDefault
setupNoSetup configuration (optional)
projectYesProject configuration
testingYesTest configuration
advancedNoAdvanced overrides (escape hatch for any HyperExecute field)
jobLabelNoLabels for the job
artifactsNoArtifact configuration (optional)
executionNoExecution settings (optional)
appTestingNoMobile app testing configuration (XCUI, Espresso or Appium). Web automation (Selenium, Playwright, Cypress, etc.) leaves it out.
sourcePayloadNoGit integration: fetches the test code from a remote Git repository. Only needed when the code is not already present locally; local code (the usual case) leaves it out.

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds modest behavioral context: smart framework defaults, input validation with security checks, and which fields are optional. It does not clarify whether output is returned as text or written to a file, so it only partly exceeds the annotation baseline.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose and required fields are front-loaded, and the bulleted framework list is scannable. There is some redundancy (appTesting is explained in the features list and again in a closing paragraph) and the feature bullets lean promotional, but overall it is well organized and appropriately sized for a 9-parameter tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a complex, deeply nested tool with no output schema, the description covers required inputs, supported frameworks, and the mobile-vs-web branch well. Its one notable gap is the return value – it never says whether the tool returns YAML text, a file path, or a validated object – which matters since there is no output schema to fill that role.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter and its nested fields. The description restates the required fields (project, testing) and the mobile/web distinction, which mirrors rather than extends the schema. Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Generates a HyperExecute YAML configuration for running tests on LambdaTest's HyperExecute platform'), which is unambiguous and distinct from every sibling (none of which produce configuration artifacts). It also names the concrete required fields, so an agent immediately knows the minimum viable call.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear conditional guidance: mobile frameworks 'ONLY use appTesting for these', and web automation 'leaves it out'. It also flags which fields are required vs optional. However, it never states when to prefer this tool over alternatives or any preconditions (e.g., that this only generates config, not runs tests), so it stops short of explicit when/when-not routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAccessibilityReportGet Accessibility ReportAInspect

Runs an accessibility scan on the given URLs and returns the issues found. Pass urls to start a scan, or testId to re-read a scan that already ran; exactly one of the two. Results arrive in issueDetails: allIssues (most-severe-first, one page at a time), rules (name and reference link per rule id, listed once), pages (every scanned page with its score and issue count), failedRules (per-rule totals for the whole scan) and pagination. Each issue carries its own howToFix and criteria, measured for that element - two issues sharing a rule can have different causes and fixes. While pagination.nextOffset is not null the results are incomplete: call again with the testId from the response plus that offset, keeping the same filters, until it is null. Page with testId, never by passing urls again - that starts a new scan, costs another run, and can return a different set of issues whose offsets do not line up. a11yReportLink is the dashboard URL and needs a login; shareableLink is public and is only returned by the original scan.

ParametersJSON Schema
NameRequiredDescriptionDefault
urlsNoList of URLs to scan. Starts a new scan. Provide either urls or testId, not both.
limitNoMaximum issues to return in this response (default 50, max 200).
offsetNoIndex to start from within the filtered, severity-sorted issue list. Pass the nextOffset from a previous response, together with that response's testId, to page through results.
testIdNoRe-read an existing scan by its testId (as returned in a previous response) instead of running a new one. Use this to page through results with offset.
pageUrlNoOnly return issues found on this scanned page. Must exactly match one of the URLs in issueDetails.pages.
severityNoOnly return issues of this impact level. Omit to get all severities.
include_rulesNoReturn the full rule catalog (about 11 KB) instead of the default ruleCatalog summary of counts per category and disabled rule ids.

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations (readOnlyHint=false, openWorldHint=true, destructiveHint=false) signal a non-read compute action with external effects; the description adds the crucial cost implication ("costs another run"), the auth distinction between a11yReportLink (needs login) and shareableLink (public, scan-only), and the pagination completion contract. This is substantial context the structured fields do not carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Purpose is front-loaded, then output shape, then pagination workflow, then link caveats. Despite its length, every sentence carries operational information (offset alignment, re-scan cost, link auth) that an agent needs; nothing is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description bears full responsibility for return-value semantics, and it delivers: it names issueDetails sub-fields (allIssues, rules, pages, failedRules, pagination), explains severity ordering and per-element howToFix/criteria, and defines the pagination termination condition. An agent has everything needed to call and consume this tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema coverage the baseline is 3, but the description adds real meaning on top: it clarifies the urls/testId exclusivity, the testId+offset pairing for paging, and that filters must stay constant across pages. It leaves some params (severity, pageUrl, include_rules) to the schema, which documents them adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ("Runs an accessibility scan on the given URLs") and immediately distinguishes its two operating modes: starting a scan via urls versus re-reading via testId. An agent understands both what it does and the dual nature of the tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the exactly-one-of constraint between urls and testId, explains when to use each, and proactively warns that re-passing urls to page starts a new scan, costs another run, and yields misaligned offsets. This is precisely the conditional guidance that prevents incorrect invocation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAnalyticsDataGet Analytics DataA
Read-only
Inspect

Summarises web automation test results in a date range: total tests, passed, failed, completed without a pass/fail mark, pass rate (passed over all tests, as the Insights page shows it), pass rate over tests marked passed or failed, average duration and distinct builds, grouped by status, browser, os, build, device or day. Computed from the automation test list (up to 5000 tests per call; the total is always exact).

ParametersJSON Schema
NameRequiredDescriptionDefault
toDateNoEnd date, YYYY-MM-DD. Defaults to today.
groupByNoDimension to group by. Defaults to status.
fromDateNoStart date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate.

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, non-destructive, closed-world), and the description usefully adds operational context: it is computed from the automation test list, capped at 5000 tests per call, yet the total is always exact, and pass rate is defined to match the Insights page. This clarifies how the numbers are derived rather than just repeating annotation hints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that opens with the verb and then lists metrics and grouping options compactly. Dense but every clause carries information; no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description carries the burden of explaining returns, and it does so thoroughly by naming each aggregated metric and the possible grouping dimensions. Minor gap: it does not describe the shape of the returned records per group, but otherwise complete for a read-only analytics tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter descriptions already document formats, defaults, and the 59-day retention limit for fromDate. The description adds no parameter-level detail beyond restating the date-range grouping, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Summarises web automation test results') and enumerates the exact metrics produced (total, passed, failed, pass rates, avg duration, distinct builds) plus the group-by dimensions. This makes it clearly distinguishable from siblings such as getTestTrends, getFlakyTests, and getAnalyticsErrors without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The date-range scope and aggregate nature imply when the tool applies, but there is no explicit when-to-use versus alternatives guidance and no mention of getTestTrends or getAnalyticsErrors as related options. Usage is left to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAnalyticsErrorsGet Analytics ErrorsA
Read-only
Inspect

Summarises failed and errored web automation tests in a date range: counts by failure category and by status, and the failing tests themselves with build and environment. Failure categories exist only for about the last 7 days of tests; older failures are counted as uncategorised.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum failing tests to list, 1 to 200. Defaults to 20.
toDateNoEnd date, YYYY-MM-DD. Defaults to today.
fromDateNoStart date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate.

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, non-destructive, non-open-world), so the description's job is to add context — and it does: the ~7-day window for failure categories and the fact older failures fall back to 'uncategorised' is a real, non-obvious data caveat. It stops short of noting ordering, latency, or empty-result behaviour.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler. The core purpose and return shape are front-loaded, and the second sentence is a genuine caveat that earns its place rather than padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the burden of describing returns and does so — counts by failure category and status, plus the failing tests with build and environment. Combined with the fully documented input schema and safety annotations, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with limit, fromDate and toDate all fully documented including defaults and the 59-day bound. The description only restates 'date range' generically and adds no format or boundary detail beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (summarises) and a tightly scoped resource (failed and errored web automation tests) plus the exact breakdowns returned, which separates it from broad analytics tools. However, it never names the neighbouring tools (getAnalyticsData, getFlakyTests, getTestTrends), so an agent must infer the distinction from scope alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the 'date range' framing and the failure-only scope let an agent infer it is for troubleshooting recent failures. There is no explicit when-to-use, no when-not-to-use, and no pointer to alternatives such as getTestTrends for pass/fail trends or getFlakyTests for intermittency.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAutomationTestBrowserConsoleLogsGet Automation Test Browser Console LogsA
Read-only
Inspect

Fetches Browser Console logs for a specific automation test. Note: Not available for mobile app automation tests.

ParametersJSON Schema
NameRequiredDescriptionDefault
testIdYesThe ID of the automation test
platformYesThe platform type: "web" for browser automation, "app" for app automation (not supported)

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered without the description. The description adds the mobile-platform limitation, which is real context, but says nothing about pagination, log retention windows, truncation, or what the returned logs look like. Useful but thin beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, zero filler, with the core purpose front-loaded and the platform caveat immediately after. Nothing could be removed without losing information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only tool this is close to adequate, but there is no output schema, so the description is the only place the return shape could be described (log entries, ordering, volume). It also omits any sibling routing for the four near-identical log tools, leaving the agent to guess.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both testId and platform are already documented in the schema, including the enum note that 'app' is unsupported. The description restates the mobile restriction but adds no new syntax or format guidance, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Fetches Browser Console logs for a specific automation test.' That distinguishes it by log type from siblings like getAutomationTestCommandLogs and getAutomationTestNetworkLogs, though those alternatives are never named. It is clear on its own but leaves the agent to infer the sibling boundaries.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives one useful exclusion ('Not available for mobile app automation tests'), which tells the agent when this tool does not apply. However, it never states when to reach for this tool versus the command/network/Selenium log siblings, so the positive selection condition is only implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAutomationTestCommandLogsGet Automation Test Command LogsC
Read-only
Inspect

Fetches command logs for a specific automation test

ParametersJSON Schema
NameRequiredDescriptionDefault
testIdYesThe ID of the automation test
platformYesThe platform type: "web" for browser automation, "app" for app automation
sessionIdNoRequired for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)
pageNumberNoPage number to fetch (default: 1)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so safety is covered. The description contributes nothing beyond that: no mention of pagination behavior, log volume/ordering, or output shape. It neither contradicts nor enriches the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence with no waste and the key resource front-loaded. It is efficient, though arguably too terse for a tool sitting in a crowded log-retrieval family.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and four parameters, the description should carry the load on what command logs actually contain, pagination expectations, and how this differs from sibling log tools. None of that is present, leaving the definition under-specified for a 4-param, paged retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including a notably useful note that sessionId is required for the app platform and comes from listMobileSessions/getAutomationTestDetails. The description adds no parameter meaning of its own, so the schema-driven baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Fetches) and resource (command logs for a specific automation test), so the basic purpose is clear. However, it does not differentiate itself from near-identical siblings such as getSessionV2CommandLogs, getAutomationTestSeleniumLogs, getAutomationTestNetworkLogs, and getAutomationTestBrowserConsoleLogs, which an agent must choose between.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no when-to-use guidance, no exclusions, and no routing to alternatives. In a cluster of four-plus log-retrieval tools this is a real hazard, since nothing tells the agent why it would pick command logs over console/network/Selenium logs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAutomationTestDetailsGet Automation Test DetailsB
Read-only
Inspect

Fetches detailed information about an automation test by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
testIdYesThe ID of the automation test
platformYesOptional: set to "app" to fetch details from the mobile API

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structured data. The description adds no further behavior beyond 'fetches details' — no note on which fields come back, whether results are cached, or the platform-routing behavior. With annotations doing the heavy lifting, this is merely adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence with the verb and resource front-loaded and zero filler. Nothing to trim.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description must at least hint at what 'details' contains — it does not. Given a crowded sibling namespace of test-fetching tools, the definition also fails to position this tool relative to them, leaving the agent under-informed for a two-required-parameter call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema itself explains both testId and the platform enum's mobile-API effect. The description contributes only the vague phrase 'by ID', adding no meaning beyond the schema, so the baseline of 3 for high coverage applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetches detailed information about an automation test by ID'), which is clear enough to act on. However, it does not distinguish itself from the many nearby siblings that also fetch automation-test data (getAutomationTestCommandLogs, getAutomationTestNetworkLogs, getAutomationTestSeleniumLogs, getAutomationTestBrowserConsoleLogs) — those are log-specific, but the description never frames this tool as the general test-metadata lookup versus those.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance at all: nothing says whether this should be called before/after the log-fetching siblings or what prompts it. The only routing hint ('set to "app" to fetch details from the mobile API') is buried in the schema, not the description. An agent has to infer the whole call context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAutomationTestNetworkLogsGet Automation Test Network LogsB
Read-only
Inspect

Fetches network logs for a specific automation test

ParametersJSON Schema
NameRequiredDescriptionDefault
testIdYesThe ID of the automation test
platformYesThe platform type: "web" for browser automation, "app" for app automation
sessionIdNoRequired for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structured data. The description adds nothing beyond the annotations — no return format, log volume, or pagination behavior — so a baseline 3 is appropriate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler or redundancy. It is efficient, though its brevity comes at the cost of any usage or behavioral detail rather than being a model of informative concision.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read-only log fetch with a fully documented 3-parameter schema and no output schema, the annotations and schema carry most of the burden. The description still omits any usage context or output expectations, leaving it only minimally adequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema itself documents testId, the platform enum, and the conditional sessionId requirement in detail. The description adds no parameter meaning beyond what the schema already provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb (fetches) and resource (network logs) scoped to a single automation test, which distinguishes it from the log siblings (browser console, command, Selenium logs) by resource type. It does not explicitly say how it differs from those siblings, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus the other automation-test log tools, and no mention of prerequisites such as the app-platform sessionId requirement. The description leaves the agent to infer the selection criteria entirely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getAutomationTestSeleniumLogsGet Automation Test Selenium LogsA
Read-only
Inspect

Fetches Selenium logs (web), Appium logs (mobile/Appium), or instrumentation logs (mobile/XCUI/Espresso) for a specific automation test. Automatically detects XCUI and Espresso tests and fetches the correct log type.

ParametersJSON Schema
NameRequiredDescriptionDefault
testIdYesThe ID of the automation test to fetch logs for
platformYesThe platform type: "web" for Selenium logs, "app" for Appium/instrumentation logs
sessionIdNoRequired for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safe read-only profile (readOnlyHint=true, destructiveHint=false). The description adds useful behavior — the three log categories and the fact that XCUI/Espresso are auto-detected — but says nothing about log size, truncation, or availability windows for a log-retrieval tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences with no filler, and the log-type mapping is front-loaded. Efficient, though the second sentence could be folded in without loss.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the return-value burden and does identify what comes back (web/Appium/instrumentation logs). It is adequate for a three-parameter read tool, missing only edge details like empty-log or unavailable-session behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including the important sessionId constraint (required for app platform, not the test ID) and the platform enum. The description adds no parameter guidance beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clear verb ("Fetches") plus a specific resource (Selenium/Appium/instrumentation logs) scoped to "a specific automation test". However, it does not distinguish itself from close siblings like getAutomationTestBrowserConsoleLogs, getAutomationTestCommandLogs, or getAutomationTestNetworkLogs, so the agent must infer which log family each tool returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied (use it to get logs for one test), and the note about auto-detecting XCUI/Espresso gives some routing help. But there is no explicit when-to-use/when-not or named alternative versus the other getAutomationTest*Logs siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getFlakyTestsGet Flaky TestsA
Read-only
Inspect

Lists tests that the insights engine marks as flaky (passing and failing intermittently) in a date range, with each test's flake rate, build and environment. Returns only flaky tests, plus how many tests were scanned. Flakiness is only computed for about the last 7 days of tests; the response says how many tests in the range had it.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum flaky tests to return, 1 to 200. Defaults to 50.
toDateNoEnd date, YYYY-MM-DD. Defaults to today.
fromDateNoStart date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate.

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish the safe, read-only, closed-world profile. The description adds genuinely useful non-annotation context: flakiness is only computed for roughly the last 7 days, only flaky tests are returned, and the payload includes a scanned-test count with how many had flakiness computed. No auth or rate-limit detail, but the disclosure goes meaningfully beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with no filler; the what-it-returns content is front-loaded and the 7-day limitation is attached directly to the flakiness concept rather than buried at the end.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description carries the return-value burden and does so well, naming flake rate, build, environment, and the scanned count. It stops short of describing ordering or empty-result behavior, but for a zero-required-parameter read tool this is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description still adds value by explaining a constraint the schema hints at only partially: the 7-day flakiness computation window versus the 59-day serviceable range, which tells the agent results will be sparse or absent for older fromDate values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Lists) and a precise resource (tests marked flaky, i.e. passing/failing intermittently) scoped to a date range, and enumerates the returned fields (flake rate, build, environment). An agent can distinguish this from report/exception siblings like getTestTrends or postTestExceptions without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The date-range scope and the 'returns only flaky tests' framing imply when to reach for this tool, and the 7-day flakiness window is a useful caveat. However, no alternative tool is named and there is no explicit when-not guidance, so usage is only implied rather than directed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteJobArtifactsGet HyperExecute Job ArtifactsB
Read-only
Inspect

Retrieve all artifacts or categorized errors for a HyperExecute job or task.

ParametersJSON Schema
NameRequiredDescriptionDefault
jobIdYesThe HyperExecute job ID
taskIdNoOptional: The HyperExecute task ID for categorized errors

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds only the two-mode behavior (all artifacts vs categorized errors); it says nothing about result volume, pagination, or how large artifact sets are returned.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with zero filler. It is efficient, though bordering on under-specified for a tool with multiple retrieval modes and several sibling artifact tools.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple read tool with annotations and no output schema, the description is minimally adequate. It does not define what 'categorized errors' contains or how this differs from downloadJobArtifact and getJobArtifactDetail, which is the main open question an agent would have.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already states that taskId is optional and drives categorized errors. The description restates this rather than adding format, constraints, or interaction details, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clear verb (Retrieve) plus resource (artifacts / categorized errors) scoped to a HyperExecute job or task. However, it does not distinguish itself from close siblings like downloadJobArtifact or getJobArtifactDetail, so an agent cannot tell which artifact-retrieval tool to pick from the description alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: passing taskId selects the 'categorized errors' mode while jobId alone returns all artifacts. There is no explicit when-to-use, no mention of alternatives such as downloadJobArtifact/getJobArtifactDetail, and no note on limits or size of results.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteJobInfoShow HyperExecute JobB
Read-only
Inspect

Fetches and displays information about a HyperExecute job including status, test results, and progress.

ParametersJSON Schema
NameRequiredDescriptionDefault
jobIdYesThe ID of the HyperExecute job

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds genuine value by enumerating returned content (status, test results, progress), which matters because there is no output schema, but it omits auth requirements, result freshness, or behavior for an unknown/invalid jobId.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence, front-loaded with the verb and the return content, with no filler. It is slightly generic in the 'Fetches and displays' opening but wastes nothing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read tool with annotations covering safety and no output schema, the description adequately conveys what the caller gets back. It falls short on sibling differentiation against the several other HyperExecute read tools and on any behavioral edge cases, leaving it minimum-viable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter with 100% schema description coverage ('The ID of the HyperExecute job'), so the schema already carries the semantics and 3 is the baseline. The description only implies that jobId selects the job and adds no format or sourcing detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Fetches and displays information about a HyperExecute job') and even names the content returned (status, test results, progress), which distinguishes it from artifact/log/session fetchers in the sibling list. It does not, however, differentiate itself from analyzeHyperExecuteJob or listHyperExecuteJobs, so an agent must still infer which HyperExecute read tool to pick.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no prerequisites, and no mention of alternatives such as analyzeHyperExecuteJob or getHyperExecuteJobSessions. The agent gets a purpose but no routing signal.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteJobLiveStreamGet HyperExecute Job Live StreamB
Read-only
Inspect

Stream live output from a running HyperExecute job. (UNDOCUMENTED - may change without notice)

ParametersJSON Schema
NameRequiredDescriptionDefault
jobIdYesThe running job ID

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/destructiveHint=false, so the safety profile is covered. The description adds useful behavioral signals: output is live/streaming rather than a one-shot fetch, and the tool is explicitly undocumented and unstable. It does not describe stream termination, blocking behavior, or error handling, so it is only partially transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the core action and immediately followed by the stability caveat. Nothing is wasted and no filler is present.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a streaming tool this is somewhat thin: it does not explain how the stream is consumed or terminated, and there is no output schema to compensate. Annotations cover the safety profile and the single parameter is fully documented, so it is minimally adequate but leaves the streaming semantics under-specified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single jobId parameter is documented as 'The running job ID' in the schema. The description adds no syntax, format, or constraint detail beyond what the schema already provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Stream live output') scoped to a 'running HyperExecute job'. The words 'live' and 'running' naturally distinguish it from siblings like getHyperExecuteJobInfo or getHyperExecuteStageLogs, though it stops short of explicitly naming those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use / when-not-to-use guidance, and no alternative is named. The only caveat, 'may change without notice,' is a stability warning rather than usage direction, so the agent must infer when a live stream is preferable to the batch log tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteJobSessionsShow HyperExecute Job SessionsC
Read-only
Inspect

Fetches and displays session information for a HyperExecute job.

ParametersJSON Schema
NameRequiredDescriptionDefault
jobIdYesThe ID of the HyperExecute job
limitNoMaximum number of sessions to return (default: 10)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds nothing beyond that — no mention of pagination via 'limit', result ordering, or whether sessions include live versus completed runs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler or redundancy. It is efficient, though the brevity reflects under-specification rather than disciplined conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter read tool with full annotation coverage and no output schema, the description is minimally adequate. It omits what a 'session' represents in this context and how results are scoped or ordered, which would help disambiguate it from the many adjacent HyperExecute readers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both jobId and the limit default of 10 are already documented in the schema. The description contributes no additional parameter meaning, which is the expected baseline when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a clear verb ('Fetches and displays') plus a specific resource ('session information for a HyperExecute job'). The resource distinguishes it from job-level siblings like getHyperExecuteJobInfo and listHyperExecuteJobs, but the description does not explicitly contrast with them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus sibling options such as getHyperExecuteJobInfo, getHyperExecuteStage, or getHyperExecuteJobLiveStream, and no stated preconditions or exclusions. The agent must infer the context from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteStageGet HyperExecute StageA
Read-only
Inspect

Retrieve the stages of a HyperExecute task (cache/prerun/scenario/etc.). Pass a stageId to return just that one stage. Stages are keyed by their parent task, so taskId is required.

ParametersJSON Schema
NameRequiredDescriptionDefault
taskIdYesThe HyperExecute task ID whose stages to fetch
stageIdNoOptional: return only the stage with this ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is carried by structured data. The description adds the structural insight that stages are keyed by parent task, but says nothing about return shape, pagination, or error behavior when a stageId is not found.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, all earning their place, with the core purpose front-loaded before the parameter guidance. No redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, two-parameter tool with full schema coverage and annotations covering safety, the description gives enough context to call it correctly. The only minor gap is that return content is undescribed, which is tolerable given there is no output schema and the resource is intuitive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine meaning beyond the schema by explaining WHY taskId is required (stages are keyed by their parent task) and by indicating the optional stageId narrows the result to a single stage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Retrieve) and resource (stages of a HyperExecute task) and clarifies the resource with examples (cache/prerun/scenario). An agent can distinguish it from getHyperExecuteTask and getHyperExecuteStageTasks, though it never names those siblings explicitly to reinforce the distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear conditional guidance for the optional stageId ('Pass a stageId to return just that one stage') and explains when taskId is required ('Stages are keyed by their parent task'). It does not name alternative tools or a when-not-to-use case, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteStageLogsGet HyperExecute Stage LogsB
Read-only
Inspect

Retrieve logs for a HyperExecute stage. (UNDOCUMENTED - may change without notice)

ParametersJSON Schema
NameRequiredDescriptionDefault
stageIdYesThe stage ID

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already establish this is a safe, read-only, non-destructive, closed-world operation, so the bar is lower. The description adds one genuinely new behavioral fact – the endpoint is undocumented and may change without notice – which is useful for an agent assessing reliability, but it omits anything about log scope, size, or pagination.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, purpose front-loaded first, with a caveat second. Nothing is padded or redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and only one parameter, the definition is minimally serviceable, but it never indicates what the returned logs contain or how they are scoped (per stage, per run). The stability warning is a nice touch but does not compensate for the absent return expectations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single stageId parameter, so the schema already carries the parameter documentation; there is only a minimal 'The stage ID' description. The tool description adds no format or sourcing detail beyond the schema, making the baseline 3 appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Retrieve) and resource (logs for a HyperExecute stage), so the purpose is unambiguous. However, it does not distinguish itself from the closely related sibling getHyperExecuteTaskLogs or clarify its relationship to getHyperExecuteStage/getHyperExecuteStageTasks, leaving an agent to guess which log-retrieval tool applies at which granularity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus the many sibling log and stage tools such as getHyperExecuteTaskLogs or getHyperExecuteJobArtifacts. The only added sentence concerns API stability, not selection criteria or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteStageTasksGet HyperExecute Stage TasksB
Read-only
Inspect

Retrieve tasks for a HyperExecute stage. (UNDOCUMENTED - may change without notice)

ParametersJSON Schema
NameRequiredDescriptionDefault
stageIdYesThe stage ID

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds the notable 'UNDOCUMENTED - may change without notice' stability caveat, which is genuine behavioral information beyond annotations, but says nothing about return contents, pagination, or auth.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single short sentence with the stability caveat appended. Front-loaded and zero waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple single-parameter read tool with annotations covering safety and a fully described schema, the core need is met. However, no output schema exists and the description doesn't say what the tasks response contains, and it omits routing guidance among several HyperExecute task/stage siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with the single stageId parameter fully documented in the schema. The description adds no syntax or format detail, so baseline 3 applies when the schema carries the load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Retrieve) and resource (tasks for a HyperExecute stage), which clearly identifies the operation. It doesn't explicitly differentiate from close siblings like getHyperExecuteTask (singular) or getHyperExecuteStage, but the resource scope is clear enough.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this versus siblings such as getHyperExecuteTask or getHyperExecuteStage. The '(UNDOCUMENTED - may change without notice)' caveat is a warning, not usage direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteTaskGet HyperExecute TaskA
Read-only
Inspect

Retrieve details for a HyperExecute task by task ID. (UNDOCUMENTED - may change without notice)

ParametersJSON Schema
NameRequiredDescriptionDefault
taskIdYesThe task ID

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this is a safe read (readOnlyHint=true, destructiveHint=false, openWorldHint=false). The description adds a genuinely useful behavioral signal not present in the structured data: the tool is UNDOCUMENTED and may change without notice, warning the agent about stability risk.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One sentence states the action plus a compact parenthetical warning. Nothing is padded and the purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read tool with full schema coverage and annotations carrying the safety profile, the description is adequate. It stops short of describing the shape of returned details or how it differs from the several similar HyperExecute retrieval tools, leaving some ambiguity in a dense sibling set.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with a single required taskId parameter documented in the schema. The description restates the task ID lookup without adding format, validation, or sourcing details, so it earns the baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Retrieve) and resource (HyperExecute task details) with the lookup key (task ID). It is reasonably distinguishable from siblings like getHyperExecuteJobInfo or getHyperExecuteStageTasks, but it never explicitly contrasts itself with those tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus the many sibling getHyperExecute* tools, and no prerequisites or context is given. The agent must infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getHyperExecuteTaskLogsGet HyperExecute Task LogsB
Read-only
Inspect

Retrieve logs for a HyperExecute task. (UNDOCUMENTED - may change without notice)

ParametersJSON Schema
NameRequiredDescriptionDefault
taskIdYesThe task ID

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description does add one genuinely useful behavioral trait beyond the annotations: the '(UNDOCUMENTED - may change without notice)' caveat warns that output/behavior is unstable. It still says nothing about log availability timing, size, or format, so it is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short clauses with zero padding, and the core purpose is front-loaded ahead of the caveat. Nothing is repeated from the title or schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-param read tool with full schema coverage and annotations, the description covers the essentials, and no output schema exists so return values need not be explained. It is still missing the sibling disambiguation and any note on when logs become available, which leaves a real gap in a crowded log-retrieval family.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single parameter with 100% schema description coverage, so the schema already documents 'taskId'. The description adds no syntax, format, or sourcing detail (e.g. where to obtain a task ID) beyond what the schema provides, matching the baseline 3 for schema-covered params.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Retrieve logs for a HyperExecute task'), so the agent knows the operation. However, it does not distinguish itself from close siblings such as getHyperExecuteStageLogs or getHyperExecuteTask, and does not clarify whether logs are task-scoped or stage-scoped, which is the key ambiguity in this family.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance, no mention of prerequisites (e.g. whether the task must be completed or still running), and no routing to alternatives like getHyperExecuteStageLogs when stage-level logs are wanted. The agent must infer usage entirely from the name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getJobArtifactDetailGet Job Artifact DetailB
Read-only
Inspect

Get details for a specific artifact within a HyperExecute job.

ParametersJSON Schema
NameRequiredDescriptionDefault
jobIdYesThe HyperExecute job ID
artifactIdYesThe artifact ID

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds no additional behavioral context such as authentication needs, pagination, or what happens for missing artifacts, but it also does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single clear sentence, front-loaded with the action and resource. Every word earns its place with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter read operation with complete schema and read-only annotations, the description is minimally sufficient. It does not explain what artifact details are returned, but no output schema exists to carry that burden either.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so both jobId and artifactId are fully documented in the schema. The description adds no further meaning to these parameters, which is acceptable given the high schema coverage baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: get details for a specific artifact within a HyperExecute job. It distinguishes itself from list-oriented siblings like getHyperExecuteJobArtifacts by emphasizing a specific artifact, though it does not explicitly name alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides no guidance on when to use this tool versus alternatives such as getHyperExecuteJobArtifacts or downloadJobArtifact. The intended context is implied only by the phrase 'specific artifact'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getLiveSessionLogsGet Live Session LogsA
Read-only
Inspect

Retrieve the request and response log of a test session, while it runs or after it ends. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe web automation test ID (e.g. DA-WIN-...) or session ID

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Adds real behavioral context beyond annotations: that app automation sessions return a not-found error and which sibling carries app-session media. Annotations already declare readOnly/openWorld=false, so the description's job is smaller, but it could still note log retention, size limits, or completeness of the live log.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly packed sentences with the core capability front-loaded, immediately followed by the platform restriction and the alternative. No filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given one parameter, no output schema, and annotations covering the safety profile, the description is nearly complete for correct invocation. The remaining gap is output shape (log format, truncation), which is minor for a read-only retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the single sessionId already documents the accepted ID format (DA-WIN-... or session ID). The description adds no format or identifier nuance beyond that, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource ('Retrieve the request and response log of a test session') with explicit scope ('while it runs or after it ends'). It explicitly distinguishes itself from mobile siblings by restricting to web automation and pointing app-session consumers to listMobileSessions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

States the exact when: web automation (Selenium, Playwright, Cypress) only, and the when-not: app automation returns not found, with the alternative listMobileSessions called out for video/screenshots. This is explicit routing, not inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getMobileCapabilityGeneratorGet Mobile Capability GeneratorC
Read-only
Inspect

Retrieve capability generation data for mobile automation test configuration.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds nothing beyond that — no note on whether the capability payload reflects live org state, whether it can fail on unconfigured accounts, or what a typical response looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler or repetition. It is appropriately sized, though its brevity comes at the cost of substance rather than from efficient editing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the burden of explaining what 'capability generation data' consists of, and it does not. For a zero-parameter getter the agent cannot infer the return shape or any gating condition from anywhere else in the definition.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to disambiguate; baseline 4 applies. No parameter-level confusion is possible.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

It states a verb (Retrieve) and a nominal resource (capability generation data for mobile automation test configuration), so the domain is identifiable. However, 'capability generation data' is undefined jargon that restates the tool name rather than explaining what is actually returned, leaving the agent unable to distinguish it confidently from siblings like getMobileConcurrency or getSupportedPlatforms.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description contains no when-to-use guidance, no prerequisites, and no mention of any alternative tool. Given the large sibling set of mobile/test-configuration getters, the absence of routing guidance is a real gap.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getMobileConcurrencyGet Mobile ConcurrencyA
Read-only
Inspect

Retrieve the current concurrency usage and limits for mobile automation (real device testing).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=false, so the safe-read profile is covered. The description adds the useful scope that the numbers are mobile/real-device specific and that it returns both usage and limits, but says nothing about whether values are a live snapshot or cached.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence with no filler; the resource and its scope appear immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description does name what comes back (usage and limits), which covers the core need for a parameterless read tool. It stops short of describing freshness, units, or per-platform breakdown.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Zero parameters, so the schema carries no semantics to supplement and the baseline is 4. Nothing in the description misrepresents the empty input contract.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Retrieve) and resource (current concurrency usage and limits) scoped to mobile automation / real device testing. It is clear, but it never explicitly distinguishes itself from the sibling getOrgConcurrency, leaving the mobile-vs-org boundary to inference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The scoping phrase 'for mobile automation (real device testing)' implies when the tool is relevant, but there is no explicit when-to-use, prerequisite, or named alternative such as getOrgConcurrency. Usage is inferable rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getOrgConcurrencyGet Org ConcurrencyB
Read-only
Inspect

Retrieve the current concurrency usage and limits for the organization on web automation (Selenium/Cypress).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the agent knows this is a safe read operation. The description adds the context that the concurrency is for web automation, which is useful. It does not, however, disclose behavioral traits beyond what annotations provide, such as whether this is a real-time lookup or if the response includes historical data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence that efficiently conveys the tool's purpose and scope with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity (no params, no output schema) and the annotations covering safety, the description is adequate. However, it lacks a crucial piece: an explicit statement that this tool is for web automation (Selenium/Cypress) and not mobile, which would help the agent choose between it and getMobileConcurrency.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description correctly implies no parameters are needed, and it clarifies the scope (organization-level, web automation) without introducing unnecessary parameter details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb 'retrieve' and resource 'current concurrency usage and limits for the organization on web automation (Selenium/Cypress)', clearly scoping it to web automation. However, it does not explicitly differentiate itself from the sibling 'getMobileConcurrency', which retrieves mobile concurrency limits.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool versus alternatives. While the tool name and description imply it should be used to get org concurrency, the presence of 'getMobileConcurrency' as a sibling creates ambiguity that is not resolved – the description does not state that this tool is for web (Selenium/Cypress) as opposed to mobile.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getSessionScreenshotsGet Session ScreenshotsA
Read-only
Inspect

Retrieve command-by-command screenshots captured during a test session. Returns screenshot URLs for each step. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe web automation test ID (e.g. DA-WIN-...) or session ID

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover safety (readOnlyHint true, destructiveHint false, openWorldHint false). The description adds meaningful behavioral context beyond that: it specifies the web-only scope, the error behavior for app sessions, and that it returns per-step screenshot URLs. It does not cover pagination or auth requirements, but for a read-only tool this is a strong addition.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with zero waste: the core action and return value are front-loaded, followed by scope constraints and the alternative tool. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description still tells the agent what to expect (screenshot URLs for each step). It also fully covers the key usage boundary (web vs app automation) and points to the sibling that handles app sessions, so an agent has all needed context to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single sessionId parameter, which already documents its expected format (e.g., DA-WIN-...) and dual meaning (test ID or session ID). The description adds no further parameter detail, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (retrieve) and resource (command-by-command screenshots captured during a test session), and immediately clarifies the output (screenshot URLs for each step). It also distinguishes itself from sibling getSessionVideo by focusing on screenshots and from listMobileSessions by confining scope to web automation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool (web automation sessions only) and when not to use it (app automation sessions return not found). It names the alternative tool for app sessions (listMobileSessions), leaving no ambiguity about routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getSessionV2CommandLogsGet Session V2 Command LogsA
Read-only
Inspect

Retrieve v2 command execution logs for a test session. Returns structured JSON with enhanced detail compared to v1 logs. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe web automation test ID (e.g. DA-WIN-...) or session ID

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is low. The description adds genuinely useful behavior beyond them: the web-only constraint, the concrete failure mode (not found) for app sessions, and the return shape ('structured JSON'). It does not mention volume, pagination, or retention limits for logs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, zero filler, and the core purpose is front-loaded before the scope constraint and the redirect to the alternative. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter log retrieval tool with no output schema, the description covers purpose, scope, and a rough return shape, which is close to sufficient. Minor gaps remain around log volume/pagination expectations, but nothing blocks correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter, and schema coverage is 100% – the schema already documents sessionId as a web automation test ID or session ID. The description adds no format nuance, examples, or validation rules beyond that, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (retrieve v2 command execution logs for a test session) and explicitly positions itself against the v1 variant by promising 'enhanced detail compared to v1 logs'. It also carves out a hard scope boundary (web automation only) that separates it from log-retrieval siblings like getAutomationTestCommandLogs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit when-not ('an app automation session returns not found') and routes the agent to the alternative for that case ('listMobileSessions carries the video and screenshot links of app sessions'). It does not, however, clarify when to prefer this over getAutomationTestCommandLogs or the other log endpoints for web sessions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getSessionV2SeleniumLogsGet Session V2 Selenium LogsA
Read-only
Inspect

Retrieve v2 Selenium/WebDriver grid request and response logs for a test session. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe web automation test ID (e.g. DA-WIN-...) or session ID

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, non-destructive, closed-world). The description adds a genuinely useful behavioral fact beyond that: app automation sessions produce a not-found result rather than logs, which prevents a wasted call. Return format and pagination are still unstated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences: the core action first, then the web-only constraint and the escape hatch to listMobileSessions. Every clause earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, so the description ideally would hint at log shape or volume, which it does not. However, it fully covers the tool's scope, the platform constraint, and the failure case, which is sufficient for a single-parameter log retrieval call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and there is a single required parameter, so the schema carries the semantic load (session ID, with the DA-WIN- example). The description only confirms 'for a test session' and adds no format or syntax detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Retrieve) and resource (v2 Selenium/WebDriver grid request and response logs) scoped to a test session. It clearly separates web automation from app automation sessions, though it does not explicitly disambiguate itself from the similarly named sibling getSessionV2CommandLogs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the when-not: app automation sessions return not found. It also names an alternative (listMobileSessions) for video and screenshot links, though it does not route the agent among the other v2/automation log siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getSessionVideoGet Session VideoA
Read-only
Inspect

Retrieve the video recording URL for a completed test session. Web automation (Selenium, Playwright, Cypress) only: an app automation session returns not found; listMobileSessions carries the video and screenshot links of app sessions.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe web automation test ID (e.g. DA-WIN-...) or session ID

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered. The description adds genuine behavioral context beyond that: it only succeeds for completed web sessions and fails with 'not found' for app sessions. It does not cover auth requirements or whether the URL is time-limited, so it lands just short of a 5.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler, front-loaded with the core action followed by the platform constraint and fallback. Every clause earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read-only lookup with no output schema, the description states what is returned (the video recording URL), the precondition (completed session, web automation), and the failure mode. Nothing an agent needs to invoke it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter and schema description coverage is 100%, so the schema already documents sessionId including its format examples. The description adds no syntax or format detail beyond that. Baseline 3 is correct.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource ('Retrieve the video recording URL for a completed test session') that an agent can immediately distinguish from sibling tools like getSessionScreenshots or getSessionV2SeleniumLogs. Scope ('completed test session') is stated up front.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly restricts usage to web automation sessions and states the negative case (app automation returns not found), then routes the agent to the correct alternative (listMobileSessions) for app sessions. This is exactly the when/when-not/alternative pattern.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getSuiteOverviewGet Suite OverviewB
Read-only
Inspect

Gets comprehensive suite overview for a project including run counts, latest status, scores, schedule information, and active calls.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesThe project ID to get suite overview for

Output Schema

ParametersJSON Schema
NameRequiredDescription
suitesNo
project_idNo
scheduled_runsNo

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is lower. The description adds what data categories are aggregated, which is useful, but does not disclose freshness, aggregation scope, or authorization requirements beyond the annotations' coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single efficient sentence that front-loads the verb and resource and packs the returned content into a compact list. No wasted clauses, though the enumerated payload detail is arguably redundant with the output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return-value details are unnecessary, and the single required parameter is fully documented in the schema. The definition is complete enough to invoke correctly; only the absence of routing context against sibling read tools keeps it from a 5.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter (projectId) and schema description coverage is 100%, so the schema fully documents it. The description adds nothing about the parameter beyond what the schema already states, which is the expected baseline at full coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clear verb ('Gets') plus specific resource ('suite overview for a project') with an enumerated list of what the overview contains (run counts, latest status, scores, schedule info, active calls). It is distinguishable from mutation siblings like triggerSuiteRun, but does not differentiate itself from read-oriented siblings such as tm_get_coverage_summary or getTestTrends.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance is given: no statement of when a suite overview is preferable to getTestTrends, tm_get_coverage_summary, or the various HyperExecute job queries. The agent must infer the use case from the description alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getSupportedPlatformsGet Supported PlatformsA
Read-only
Inspect

Lists the OS, browser and device combinations LambdaTest supports, to find valid os, browser and version values for a test configuration. With no filters it returns a compact summary: each desktop OS with its browsers (version count, newest and oldest) and resolutions, and each mobile platform with its device count and OS versions. Set os (and optionally browser) for the full version list, or os=ios / os=android for device names.

ParametersJSON Schema
NameRequiredDescriptionDefault
osNoOS name to expand, matched case-insensitively as a substring, e.g. "Windows 11", "Sonoma", "android"
typeNoOnly desktop OSes or only mobile platforms
browserNoBrowser to expand on desktop, e.g. "chrome", "firefox", "edge", "safari"

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so safety is covered. The description adds real behavioral context the annotations don't: the exact shape of the unfiltered response (per-OS browser counts, newest/oldest versions, resolutions; per-mobile device counts and OS versions) and how filters change the output.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads what the tool returns, then the filter modes, with no filler sentences. The second and third sentences are dense (nested counts, resolutions, version ranges), but every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by describing the return shape in each mode, and annotations cover the safety profile. Missing only minor operational details such as result size or any cap on the unfiltered summary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so per-parameter meaning is already documented and the baseline is 3. The description goes beyond it by explaining parameter interaction — that os+browser yields a full version list while os=ios/os=android yields device names — which the schema's individual field descriptions do not convey.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Lists the OS, browser and device combinations LambdaTest supports') plus the downstream purpose ('to find valid os, browser and version values for a test configuration'). This distinguishes it from nearby siblings like listDevicesForTesting and getMobileCapabilityGenerator, which serve different lookups.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly explains the three invocation modes: no filters for a compact summary, os (optionally browser) for full version lists, os=ios/os=android for device names. That is strong guidance on how to call it, but it never states when to prefer this tool over listDevicesForTesting or getMobileCapabilityGenerator.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

getTestTrendsGet Test TrendsA
Read-only
Inspect

Daily trend of web automation test results in a date range: for each day, total tests, passed, failed, completed without a pass/fail mark, pass rate (passed over all tests), pass rate over tests marked passed or failed, and average duration.

ParametersJSON Schema
NameRequiredDescriptionDefault
toDateNoEnd date, YYYY-MM-DD. Defaults to today.
fromDateNoStart date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate.

TDQS

A3.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. With no output schema present, the description carries real weight by disclosing the returned metric set and, importantly, clarifying that 'completed without a pass/fail mark' is a distinct bucket and that two different pass-rate definitions are returned — a genuinely ambiguous point for an agent interpreting results. It stops short of stating retention/aggregation behavior beyond what the schema already says.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence that names the resource and scope first, then enumerates outputs. The list is dense but every clause maps to a distinct returned value, so nothing is filler; it could be broken up for scannability but is not padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-optional-parameter read tool with no output schema, the input side is fully covered by the schema and the output side is covered by the description's metric enumeration. What is missing is orientation — whether this is suite-wide or scoped to a project/test set, and how it relates to the other analytics tools — which keeps it short of complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both optional parameters fully documented including formats, defaults, and the 59-day retention limit. The description adds nothing about fromDate/toDate, so this is the baseline 3 case where the schema does all the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Daily trend of web automation test results in a date range') and enumerates exactly what each day's bucket contains. It is unambiguous on its own, but it never contrasts itself with adjacent analytics tools such as getFlakyTests, getAnalyticsData, or getSuiteOverview, so sibling differentiation is left to the caller.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no when-to-use guidance at all: no mention of when trend data beats a point-in-time report, no exclusions, and no alternatives named among the ~70 siblings. The date-range expectation is only inferable from the parameter names, not stated as usage guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listDevicesForTestingList Devices for TestingA
Read-only
Inspect

Lists the real devices available on LambdaTest for mobile app testing, 50 per page by default. Filter by region, OS, device name or OS version, and page with limit and offset. The text gives the total that matched.

ParametersJSON Schema
NameRequiredDescriptionDefault
osNoOperating system filter
limitNoDevices per page, 1 to 200 (default 50)
offsetNoDevices to skip (default 0)
regionNoDevice region (default: us)
deviceNameNoDevice name to match, case-insensitively as a substring, e.g. "Galaxy S24", "iPhone 15", "Pixel"
platformVersionNoOS version to match, e.g. "14" (matches 14 and 14.x)

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds real value beyond that: the default page size of 50 and the fact that the response text reports the total number matched, which is the only source of return-shape information since there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences with zero filler: what it lists, the default page size, the filterable dimensions, and the returned total. The resource and default are front-loaded so the agent gets the essential facts immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only paginated list tool with a fully documented schema and no output schema, the description covers pagination defaults, filter dimensions, and the returned total count. It is nearly complete, lacking only a note on ordering or how to interpret the matched-total value.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and every parameter is documented in the schema, including the substring/case-insensitive match semantics and the '14 matches 14.x' version behavior. The description only restates those same filters, so the baseline of 3 is appropriate with no added syntax or format detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Lists the real devices available on LambdaTest for mobile app testing', which clearly identifies a device-catalog read. It does not distinguish itself from the closest sibling (getSupportedPlatforms), which also concerns available platforms, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: the description explains that you can filter by region/OS/name/version and paginate, which hints at the discovery/search scenario. But it never says when to prefer this tool over getSupportedPlatforms or other listing siblings, and offers no exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listHyperExecuteJobsList HyperExecute JobsA
Read-only
Inspect

List the account's HyperExecute jobs, newest first, with offset pagination and an optional status filter. The status filter looks at the 200 most recent jobs.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMaximum number of jobs to return, 1 to 200 (default 10)
offsetNoNumber of jobs to skip, after the status filter (default 0)
statusNoOnly jobs with this status (e.g. completed, failed, running, skipped), among the 200 most recent

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint, destructiveHint=false, openWorldHint=false). The description adds behavior beyond that: results are newest-first, pagination is offset-based, and critically the status filter only ranges over the 200 most recent jobs, which would otherwise cause silently incomplete results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, with the core scope front-loaded and the filter limitation placed immediately after so it is read before the agent commits to a query.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still gives an agent enough to call the tool correctly: scope, ordering, pagination and the filter window. It stops short of indicating what a returned job record contains, but that is a minor omission for a straightforward list endpoint.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters are already documented in the schema, making 3 the baseline. The description's restatement of the 200-job filter window duplicates the status parameter description rather than adding new syntax or format detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List the account's HyperExecute jobs') plus the ordering and pagination model, which cleanly separates it from the many single-job siblings like getHyperExecuteJobInfo and getHyperExecuteJobSessions. It does not explicitly name an alternative for the job-detail use case, but the list scope is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied rather than stated: an agent can infer this is for browsing jobs rather than inspecting one, and the 'status filter looks at the 200 most recent jobs' caveat is a genuinely useful boundary. However, there is no explicit when-to-use vs when-not, nor a pointer to the detail/artifact siblings for drilling into a specific job.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listMobileBuildsList Mobile BuildsA
Read-only
Inspect

Lists mobile app automation builds, newest first, 10 per page by default. Page with limit and offset; the result gives the total the API reports. The API returns only builds from the last 90 days, so total can be higher than the builds you can page through.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoRows per page, 1 to 100 (default 10)
offsetNoRows to skip (default 0)
statusNoOnly builds with this status, e.g. completed, running, passed, failed

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structure. The description adds genuinely non-obvious behavior beyond that: newest-first ordering, a 10-per-page default, and most importantly that the API only returns builds from the last 90 days so the reported total can exceed what is pageable. That retention/consistency caveat is exactly the kind of disclosure annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, no filler, and the ordering/default-page-size facts are front-loaded before the caveat. Every sentence carries distinct information: what is listed, how to page, and the retention limitation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the burden of describing returns, and it does: ordering, page size, and the behavior of the total relative to the pageable set. Combined with full schema coverage on all three parameters and safe-read annotations, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so limit and offset are already documented and the baseline would be 3. The description goes slightly beyond the schema by explaining the total-vs-pageable interaction that arises from the 90-day truncation, which affects how an agent should interpret paging results. The status parameter is not addressed at all in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource ('Lists mobile app automation builds') plus ordering ('newest first') and default page size, which is far more than a restated title. It does not, however, name or implicitly distinguish itself from nearby siblings such as listMobileSessions, listUploadedApps, or listHyperExecuteJobs, leaving the resource name alone to do the disambiguation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit when-to-use / when-not-to-use statement or reference to an alternative tool for retrieving build data, so tool selection is left to inference from the resource name. The description does give practical usage mechanics (paging with limit/offset) and a conditional caveat about the 90-day window, which is useful but not selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listMobileSessionsList Mobile SessionsA
Read-only
Inspect

Lists mobile app automation sessions (tests), newest first, 10 per page by default, with their video, screenshot and log links. Page with limit and offset, or filter by buildId or status; the result gives the total the API reports.

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoRows per page, 1 to 100 (default 10)
offsetNoRows to skip (default 0)
statusNoOnly sessions with this status, e.g. passed, failed, completed, error
buildIdNoOnly sessions of this build (build_id from listMobileBuilds)

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnlyHint=true, destructiveHint=false, openWorldHint=false), so the bar is lower. The description still adds genuine behavioral context: newest-first ordering, a 10-per-page default, and that video/screenshot/log links are included in each row.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two dense, front-loaded sentences that lead with what is listed and then cover ordering, paging, filtering, and return contents. No filler and no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully tells the agent the response carries video/screenshot/log links and a total count, and it explains the pagination contract. It is essentially complete for a simple read-only list tool, though ordering ties and auth expectations are unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all four parameters including defaults and the buildId provenance. The description only restates that limit/offset page and buildId/status filter, adding no syntax or format detail beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Lists) and resource (mobile app automation sessions/tests), and adds useful scope detail such as ordering and default page size. It does not explicitly name a near sibling (e.g., listMobileBuilds or getHyperExecuteJobSessions) to disambiguate, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains how to use the tool (page with limit/offset, or filter by buildId or status) but never states when to choose it over alternatives or any prerequisites. Usage is implied rather than directed, which is minimum-viable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listTunnelsList TunnelsA
Read-only
Inspect

Retrieve all active LambdaTest tunnel connections for the account. Useful for checking which tunnels are running before starting a new one or debugging local testing connectivity.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered by structured data. The description adds the 'active' scoping qualifier and intent context, but says nothing about result size, pagination, or what an account with no tunnels returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, zero filler, with the core action front-loaded before the use-case rationale. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless read-only listing tool whose annotations already cover the safety profile, this is nearly complete. The only mild gap is that with no output schema present, the description does not indicate the shape or fields of the returned tunnel entries.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so there is nothing for the description to compensate for; the baseline for a 0-parameter tool is 4. The description correctly implies account-scoped retrieval with no input needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — 'Retrieve all active LambdaTest tunnel connections for the account' — which is unambiguous. It does not explicitly distinguish itself from related tunnel siblings such as stopTunnel or analyzeAppViaTunnel, so it falls just short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete usage contexts: checking running tunnels before starting a new one, and debugging local testing connectivity. However, it names no alternative tools and states no exclusions, so it is clear context without full when/when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

listUploadedAppsList Uploaded AppsA
Read-only
Inspect

Retrieve all apps uploaded to LambdaTest for mobile testing. Use type=android/ios for real devices, emulator/simulator for virtual devices.

ParametersJSON Schema
NameRequiredDescriptionDefault
typeNoApp type: android/ios for real devices, emulator/simulator for virtual
levelNoScope: user (default, apps you uploaded) or team (apps across your organization; sent to the API as level=org)

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, so the safety profile is covered and the bar is lower. The description adds the scope of what is listed (all uploaded mobile apps) but says nothing about result size, pagination, or whether all items are returned at once.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, with the core purpose front-loaded and the parameter hint following. Every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a read-only, two-optional-parameter list tool with no output schema and full annotation coverage, this is close to sufficient: purpose, platform scope and enum semantics are all present. Minor gaps remain around result size/pagination and how the returned apps relate to downstream tools like analyzeAppViaTunnel.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and both enums are fully documented in the schema, including the level=org mapping that the description omits entirely. The description's type sentence largely restates the schema's own wording, so it adds little beyond structured data — baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (retrieve all uploaded mobile-testing apps) with the platform named, which distinguishes it from siblings like listMobileBuilds or listMobileSessions. It does not explicitly name or contrast with any sibling (e.g. upload_app, deleteUploadedApps), so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives real selection guidance for the type parameter (android/ios = real devices, emulator/simulator = virtual), which is genuine usage help. However there is no guidance on when to use this tool versus alternatives, and the level parameter is never mentioned, so usage remains implied rather than explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

postSessionExceptionsPost Session ExceptionsAInspect

Upload exception/error logs from the client machine for a test session, by session ID. Use this to attach assertion failures or unexpected errors to a session. The result reads the test's exceptions back to confirm they were stored. The automation dashboard may not show exceptions uploaded this way after the test ended.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataYesThe exceptions to attach: {"exception": ["first message", "second message"]}
sessionIdYesThe session ID

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare this as a non-read-only, non-destructive, non-open-world write. The description adds genuinely useful behavior beyond that: the result reads exceptions back to confirm storage, and — critically — that the automation dashboard may not reflect exceptions uploaded after the test ended. That delayed-visibility caveat is real operational context the annotations cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four sentences, front-loaded with the core action and scope, and each subsequent sentence adds distinct value (use case, verification behavior, caveat). Minor slack, but no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description usefully explains the read-back confirmation behavior, and annotations cover the safety profile. For a nested-parameter mutation tool this is close to complete; permission requirements for uploading to an arbitrary session are the only noticeable omission.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the nested `data.exception` structure is fully documented with an example, so the schema carries the parameter semantics. The description only restates 'by session ID' and adds no format or constraint details beyond what the schema provides; baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: 'Upload exception/error logs from the client machine for a test session, by session ID.' The session scoping implicitly separates it from the sibling postTestExceptions, which handles test-level exceptions, though that sibling is never named explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear when-to-use guidance: 'Use this to attach assertion failures or unexpected errors to a session.' However, it offers no explicit when-not condition or named alternative (e.g. postTestExceptions for the test-level case), so the agent must infer the split.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

postTestExceptionsPost Test ExceptionsAInspect

Upload exception logs for a specific test (by test ID) rather than by session ID. The result reads the test's exceptions back to confirm they were stored. The automation dashboard may not show exceptions uploaded this way after the test ended.

ParametersJSON Schema
NameRequiredDescriptionDefault
dataYesThe exceptions to attach: {"exception": ["first message", "second message"]}
testIdYesThe test ID

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations cover the safety profile (readOnlyHint=false, destructiveHint=false, openWorldHint=false), so the description only needs to add behavior beyond that — and it does, disclosing that the call reads the exceptions back as a storage confirmation and that dashboard visibility may be unreliable post-test. It does not cover idempotency, overwrite semantics, or auth requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the action and the distinguishing scope, followed by verification behavior and the one operational caveat. No sentence is redundant or padded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter mutation with annotations covering safety and a fully documented nested schema, the definition supplies the distinguishing scope, the read-back behavior, and the post-test visibility caveat. The read-back partially compensates for the absence of an output schema, though the exact response shape remains unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the nested data object, its required 'exception' array, and item constraints are fully documented in the schema. The description only restates that identification is by test ID, adding no syntax or format detail beyond the structured fields. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (upload exception logs) with an explicit scope qualifier ('for a specific test (by test ID) rather than by session ID'). This directly distinguishes it from the sibling postSessionExceptions without needing to open either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a clear selection rule against the session-based alternative and a caveat about when the result may be misleading ('automation dashboard may not show exceptions uploaded this way after the test ended'). It does not name the sibling tool explicitly or state prerequisites such as whether the test must already exist.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smartui_baseline_manageSmartUI Baseline ManageA
Destructive
Inspect

Merge SmartUI baselines across branches or builds (to read baseline settings, use smartui_project_query with action=get). merge_branch/merge_build resolve source and target names to builds, merge them, then poll the merged build and report its real terminal status — a merge that lands in error or with zero screenshots is returned as a tool error, not as success. Merging permanently changes baseline state: the merged build takes the baseline flag, so a failed merge leaves the baseline on an empty build.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesmerge_branch merges one branch baseline into another; merge_build merges one build into another
sourceNoSource branch (merge_branch) or build name (merge_build) to merge FROM
targetNoTarget branch (merge_branch) or build name (merge_build) to merge INTO
project_idYesProject id (required)

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the destructiveHint annotation by disclosing the merge lifecycle: names resolve to builds, the merged build is polled, terminal status is reported, and error/zero-screenshot outcomes surface as tool errors rather than success. Critically warns that the merged build takes the baseline flag, so a failed merge leaves the baseline on an empty build — a non-obvious destructive consequence.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with purpose and the read alternative before behavioral detail. Dense but every clause (polling, error semantics, baseline-flag side effect) carries information an agent needs. Slightly packed, but no waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive, multi-mode mutation tool with no output schema, the description covers what the tool does, when to use it vs the read sibling, how success/failure is signaled, and the lasting state change. Nothing an agent needs to call it safely is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (baseline 3), but the description adds meaning: source/target are names that get resolved to builds, and merge_branch vs merge_build reinterpret those same fields as branches or build names. This clarifies the enum-conditional parameter semantics beyond the schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (merge) and resource (SmartUI baselines) with scope (across branches or builds), and explicitly distinguishes itself from the read-path sibling smartui_project_query. An agent can identify this as the mutation counterpart to the SmartUI query tools without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent: 'to read baseline settings, use smartui_project_query with action=get', and names the two merge modes (merge_branch/merge_build) with their distinct source/target semantics. When-to-use and the alternative for the read case are both stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smartui_build_querySmartUI Build QueryA
Read-only
Inspect

List SmartUI builds in a project, get one build, or get the CI gate verdict for a build. Use action=gate in pipelines: it returns verdict (passed|failed|pending_approval|running|error), totals (including missing screenshots the baseline has), blocking_comparison_ids, and blocking[] with per-item detail. verdict=failed covers rejections AND builds that dropped baseline screenshots. Read-only.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (list only, 1-based)
limitNoPage size (list only, default 50, max 200)
actionYeslist builds, get one build, or gate (CI verdict)
branchNoFilter builds by branch (list only)
build_idNoBuild id — required for get/gate (or use build_name)
build_nameNoBuild name. get/gate: exact name, an alternative to build_id. list: case-insensitive substring filter, applied client-side to the newest 200 builds (upstream has no name filter).
project_idYesProject id (required)
build_statusNoSubstring filter on build status (list only), e.g. "pending-approval"

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, so the trailing "Read-only" only confirms them. The description earns credit for disclosing semantics annotations cannot: the full verdict enum, that totals include missing baseline screenshots, that blocking[] carries per-item detail, and notably that verdict=failed conflates rejections with dropped baseline screenshots.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the three operations, then the high-value gate return contract, with no filler sentences. Minor waste is repeating "Read-only" already asserted by readOnlyHint=true annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly compensates for the gate path by enumerating verdicts and blocking fields. However, list and get return shapes and pagination behavior go unmentioned, leaving an agent guessing about the other two action paths of this 8-parameter tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, including action enum meanings, build_id/build_name dual usage, and the client-side substring caveat, so the schema already carries parameter semantics. The description only re-expresses action=gate usage and adds no syntax or constraint beyond the schema — baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The first sentence names the resource (SmartUI builds) and all three distinct verbs/operations (list, get, gate verdict), so an agent can map action values to intent without opening the schema. The resource itself separates it from smartui_project_query and smartui_comparison_query siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

"Use action=gate in pipelines" gives an explicit context that selects one of the three actions, which is real routing guidance. It stops short of saying when NOT to use this tool or naming SmartUI siblings (e.g. baseline/comparison tools) as alternatives for related questions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smartui_comparison_analyzeSmartUI Comparison AnalyzeA
Read-only
Inspect

Analyze one SmartUI comparison through a lens: consolidated (default), pixel, or dom. Returns the comparison data (mismatch %, verdict, correctly-labelled asset URLs) plus, for lens=dom, a real DOM byte delta when the artefact is served — no local filesystem needed. consolidated and pixel report upstream numbers and point at the assets; they do not run their own image analysis. lens=dom degrades to a structured error when DOM artefacts are not served. Read-only.

ParametersJSON Schema
NameRequiredDescriptionDefault
lensNoAnalysis lens (default consolidated). layout and perceptual were retired — they returned the consolidated payload unchanged.
comparison_idYesComparison id (required) — from smartui_comparison_query
include_assetsNourls (default) returns asset URLs only. inline additionally attaches the screenshots themselves as image blocks — compared only for lens=pixel, baseline+current+compared for consolidated, none for lens=dom. Images over 1.5 MB (or past 2.25 MB in total) are skipped and listed with the reason; their URLs stay in the result. Use inline when you cannot fetch URLs.

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/destructiveHint, and the description reinforces that while adding real behavioral detail beyond them: consolidated/pixel do not run their own image analysis, lens=dom degrades to a structured error when artefacts are absent, and no local filesystem is required. Failure and degradation modes are disclosed, which is exactly the added value the annotation does not carry.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action and lens list, then layers consequences. Dense but every clause carries information; losses come only from mild repetition of the lens behavior in two sentences.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must describe returns, and it does: mismatch %, verdict, labelled asset URLs, and the DOM byte delta for lens=dom. Combined with the error-degradation note, an agent has enough to call it correctly; only the exact error/response shape remains unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters including the inline size limits, so the baseline is 3. The description mostly restates the lens semantics already present in the schema and adds no new format or syntax guidance for comparison_id or include_assets.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Analyze) and resource (one SmartUI comparison) with an explicit scope limiter (one, not many), and names the three lens modes. It is clearly distinguishable from the sibling smartui_comparison_query, whose output feeds the required comparison_id.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains when each lens matters (dom gives a real byte delta and needs served artefacts; consolidated/pixel merely surface upstream numbers) and gives an actionable condition for include_assets=inline ('when you cannot fetch URLs'). It stops short of stating when to prefer this tool over smartui_comparison_query.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smartui_comparison_querySmartUI Comparison QueryA
Read-only
Inspect

List all comparisons in a SmartUI build (each row returns comparison_id AND ss_id), or get one comparison's detail (mismatch %, status, correctly-labelled baseline/current/compared asset URLs). This is how you obtain a comparison_id — you never need the dashboard. Read-only.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (list only, 1-based)
limitNoPage size (list only, default 50, max 200)
actionYeslist comparisons in a build, or get one by comparison_id
statusNoFilter by status (list only), e.g. "changes_found", "approved"
build_idNoBuild id — required for action=list
project_idNoProject id — required for action=list
min_mismatchNoOnly comparisons with mismatch_percentage >= this (list only)
comparison_idNoComparison id — required for action=get

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered; the description reinforces this with 'Read-only' and adds real value by enumerating return contents (comparison_id AND ss_id for list; mismatch %, status, labelled baseline/current/compared URLs for get). It does not discuss pagination behavior, which would have helped.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Dense and front-loaded: the primary list/get capability comes first, followed by the key value proposition and a trailing safety note. The standalone 'Read-only.' repeats the annotation, a minor redundancy, but nothing else is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 parameters, no output schema, and only readOnly annotations, the description does the necessary work of summarizing what each action returns. It is nearly complete for agent selection; only pagination/filtering behavior and its relationship to smartui_comparison_analyze are left unstated.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with each of the 8 parameters already documented in the schema including the list-only markers and the list/get enum. The description adds no parameter-level detail (formats, defaults, edge cases) beyond what the schema provides, so the 3 baseline applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource (comparisons) with two clear modes (list in a build vs get one), plus the fields each returns. It is easy to distinguish from the other smartui_* siblings by name, but it never explicitly differentiates itself from smartui_comparison_analyze, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It hints at intent with 'This is how you obtain a comparison_id — you never need the dashboard', which tells the agent why to call it. However, it gives no when-not guidance and does not route the agent between this tool and the nearby smartui_comparison_analyze or smartui_build_query alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smartui_project_querySmartUI Project QueryA
Read-only
Inspect

List SmartUI projects, or get one project with its settings — action=get returns baseline_branch, approvers[], smart_baseline, auto_approval_branch, overwrite_screenshot and project_category alongside platform and project_type. Start here when you do not have a project_id. Read-only.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (list only, 1-based)
limitNoPage size (list only, default 50, max 200)
actionYeslist all projects, or get one by project_id
searchNoCase-insensitive substring filter on project name (list only)
project_idNoProject id — required for action=get

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds non-obvious behavioral detail: exactly which settings fields (baseline_branch, approvers[], smart_baseline, etc.) come back from action=get. That return-shape disclosure is genuine value beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One front-loaded sentence that leads with the list/get split and closes with the routing hint. The field enumeration is slightly long but earns its place given there is no output schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description compensates by naming the fields action=get returns, and it flags read-only behavior. Pagination defaults and the full parameter set are left to the schema, which is acceptable since coverage is complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so list-only/page/limit/search/project_id semantics are already documented in the schema. The description only echoes action=get's purpose and its returned fields, adding little parameter meaning beyond the structured data. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States specific verbs (list / get) and the resource (SmartUI projects), and enumerates the fields returned by action=get. An agent can distinguish this from sibling smartui_build_query and smartui_baseline_manage without opening any schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly gives a usage cue: 'Start here when you do not have a project_id.' This tells the agent when to reach for this tool first. It does not name an alternative tool or state exclusions, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smartui_screenshot_reviewSmartUI Screenshot ReviewA
Destructive
Inspect

Approve or reject SmartUI comparisons — individually (comparison_ids or ss_ids), or for a whole build (all_matching:{status}). Ids are validated against the build before writing, and results are verified by re-reading status, so the response reports real per-item updated[]/failed[] and whether the baseline was promoted. Approving comparisons on the baseline branch can promote the build to baseline.

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesapprove or reject the selected comparisons
ss_idsNoScreenshot-variant ids to review directly
build_idYesBuild id the comparisons belong to (required)
project_idYesProject id (required)
all_matchingNoReview every comparison in the build matching a status
comparison_idsNoComparison ids to review (resolved to ss_ids server-side)

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only supply readOnlyHint=false and destructiveHint=true; the description adds substantive behavior beyond that: ids are validated against the build before writing, results are verified by re-reading status, the response reports real per-item updated[]/failed[], and approving on the baseline branch can promote the build to baseline. It also discloses that comparison_ids are resolved server-side.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tightly packed sentences with no filler; the operation and its two modes are front-loaded. Slightly dense (multiple ideas per sentence), but every clause carries information an agent needs.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates by describing the response shape (per-item updated[]/failed[]) and the baseline-promotion side effect. Combined with full schema coverage of all 6 parameters, an agent has everything required to invoke and interpret the call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds meaning the per-parameter blurbs do not: that comparison_ids and ss_ids are two alternative individual targeting paths, and that all_matching:{status} is a bulk selector over the whole build. The relationship between the targeting parameters is clarified rather than merely restated.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb pair (approve/reject) on a specific resource (SmartUI comparisons) and enumerates the two operating modes (individual ids vs whole-build all_matching:{status}). An agent can distinguish this from smartui_comparison_analyze or smartui_baseline_manage without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Clearly explains when to use each targeting mode: individual comparison_ids/ss_ids versus bulk all_matching by status. It does not name rival siblings (baseline manage, comparison query/analyze) or state exclusions, so it stops short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

smartui_screenshot_uploadSmartUI Screenshot UploadA
Destructive
Inspect

Upload screenshots to a SmartUI project as a new build, from local file_paths or from source_urls (fetched server-side, so it works without a client filesystem). Pass project_id to auto-resolve the project token. Set mark_baseline=true to seed a baseline. Returns build_id; follow with smartui_build_query(action=gate).

ParametersJSON Schema
NameRequiredDescriptionDefault
actionYesWhat to upload (currently images)
build_nameNoName for this build (optional)
file_pathsNoAbsolute local image paths (.png/.jpg/.jpeg). Only usable when the MCP server runs locally over stdio — the hosted server cannot see your filesystem; use source_urls there.
project_idNoProject id — preferred; the projectToken is resolved automatically
source_urlsNoPublic http/https image URLs, fetched server-side (max 20). Works in hosted mode.
mark_baselineNoMark this build as the baseline
screenshot_nameNoName this screenshot (binds it to that baseline) instead of using the file/URL name. Sent exactly as given, with no extension added, so pass the baseline name as it appears in SmartUI (SDK and CLI captures have no extension). Only valid when uploading exactly one image.

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false and destructiveHint=true, so the agent knows this mutates state. The description adds the build_id return and the gate follow-up, which the annotations do not convey. However, it never explains the destructive side — e.g. that mark_baseline=true may overwrite an existing baseline — which is the most important behavioral caveat for a tool flagged destructive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences, correctly front-loaded with the core action and the highest-value caveat (server-side fetch vs client filesystem). Slight redundancy: the 'fetched server-side' note repeats the schema's 'Works in hosted mode'.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly discloses the return value (build_id) and the next step, so an agent can chain into the gate query. The only gap is behavioral — what marking a baseline displaces — which a destructive-flagged mutation tool should mention.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents every parameter, including the stdio-vs-hosted constraint and the 'no extension added' rule for screenshot_name. The description largely restates this, adding only the minor tie between mark_baseline and later screenshot_name binding. Baseline 3 applies when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Upload screenshots to a SmartUI project as a new build') and covers both ingestion modes, file_paths and source_urls. This distinguishes it cleanly from siblings like smartui_screenshot_review and smartui_build_query, which operate on existing builds.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives conditional guidance: file_paths only works over local stdio, source_urls for hosted mode, project_id for token auto-resolution, mark_baseline to seed a baseline. It also names the follow-up tool (smartui_build_query(action=gate)). It stops short of saying when to use this vs smartui_baseline_manage for baseline operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stopBuildStop Automation BuildA
DestructiveIdempotent
Inspect

Stops every running test in a LambdaTest web automation build. Running sessions end immediately.

ParametersJSON Schema
NameRequiredDescriptionDefault
buildIdYesID of the build to stop

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, idempotentHint=true, and readOnlyHint=false, so the safety profile is covered. The description adds the concrete blast radius – every running test stops and running sessions end immediately – which is meaningful context for a destructive operation beyond what the annotations state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences with no filler, and the scope-defining clause (every running test in a build) is front-loaded ahead of the effect.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter destructive stop tool with no output schema, the description covers purpose, scope, and immediate effect. It could note error behavior (e.g., already-finished builds) or what is returned, but nothing critical for correct invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is a single buildId parameter with 100% schema description coverage, so the schema fully documents it. The description adds no format, type, or constraint detail beyond the schema, which is the expected baseline when the schema does the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (stops) and resource (every running test in a LambdaTest web automation build), and scopes it to the whole build rather than a single session, which distinguishes it from sibling stopSession. An agent can select it without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the verb and scope, but the description never states when to prefer this over stopSession or deleteBuild, nor any prerequisite conditions. It is adequate but gives no explicit routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stopSessionStop SessionB
Destructive
Inspect

Stop an active running test session on LambdaTest.

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYesThe session ID to stop

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and readOnlyHint=false, so the safety profile is covered. The description adds only the state constraint ('active running'), which is useful, but discloses nothing about what happens to the session's data or whether the action is reversible.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One tight sentence with no filler and the action front-loaded. Nothing wastes space, though it is arguably too terse to be maximally useful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter stop operation whose destructive nature is carried by annotations, and with no output schema to explain, the description is nearly sufficient. It only lacks routing against the similar deleteSession/stopScreenshotTest siblings.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With a single parameter at 100% schema description coverage, the schema fully documents sessionId. The description adds no format, source, or example detail beyond what the schema already provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Stop') and resource ('active running test session on LambdaTest'), so the agent knows the target and platform. It does not distinguish itself from close siblings like deleteSession or stopScreenshotTest, but the core purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this versus deleteSession (permanent removal) or stopScreenshotTest. The word 'active running' weakly implies the session must be live, but no alternatives or preconditions are named.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

stopTunnelStop TunnelA
Destructive
Inspect

Terminate a specific active tunnel connection by tunnel ID. Use this to clean up tunnels that are no longer needed.

ParametersJSON Schema
NameRequiredDescriptionDefault
tunnelIdYesThe ID of the tunnel to stop

TDQS

A3.6/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, readOnlyHint=false, and openWorldHint=false, so the safety profile is covered. The description reinforces the terminal nature with 'Terminate' and the cleanup use case, but adds no new behavioral detail such as irreversibility or effect on sessions using the tunnel.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with the action and key identifier front-loaded, followed by the use case. No filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter destructive tool with full annotation coverage, the description is essentially complete. Only minor omissions remain, such as whether stopping a tunnel affects in-flight sessions, and there is no output schema to require explanation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a single, fully documented parameter, so the schema already carries the meaning. 'by tunnel ID' merely restates what the schema says; baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Terminate) and resource (a specific active tunnel connection) keyed by tunnel ID. It is clearly distinguishable from listTunnels and the analyze/build tunnel tools, though it does not name any sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The second sentence gives a use case ('clean up tunnels that are no longer needed'), which implies when to call it, but there is no explicit when-not guidance or reference to the alternative (e.g., listTunnels to find IDs). Adequate but with clear gaps.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_create_folderCreate FolderAInspect

Create a folder — in a project's Test Cases tree (default) or Test Runs tree (pass project_id), or in the organization-level Modules tree (no project_id needed; module folders are shared across projects). Optionally nest it under an existing parent folder to build folder trees. Deleting folders is not available through this connection.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesFolder name
treeNoWhich folder tree to create in (default test-cases). "modules" is organization-level.
project_idNoProject ID (ULID) — required for the test-cases and test-runs trees; not used for modules
descriptionNoOptional description
parent_folder_idNoNest under this existing folder (ULID); omit for top level

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false, openWorldHint=false, so the agent knows this is a contained, non-destructive write. The description adds meaningful context beyond that: modules live at organization level and are shared across projects, and deletion is unsupported. It does not say what happens if the name collides or if parent_folder_id is invalid, but the added context is solid.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three compact sentences, front-loaded with the create action and the tree-choice branching, followed by nesting and the deletion caveat. No filler and no restatement of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema, the description covers the branching modes, the required nesting rules, and an important negative capability (no delete). It stops short of explaining return behavior or failure modes for invalid parent/project ids, which is a minor gap given the clean schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real semantics: which tree the default applies to, that project_id is inapplicable for modules, and that omitting parent_folder_id means top level. That is useful reinforcement and clarification rather than pure duplication.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Create) plus resource (folder) and immediately enumerates the three distinct trees it can operate on, with the default called out. This differentiates it cleanly from siblings like tm_list_folders, tm_update_folder, and tm_create_project.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the conditions that select each mode: pass project_id for Test Cases/Test Runs trees, omit it for organization-level Modules, and optionally nest via parent_folder_id. It also sets a boundary by noting that deletion is not available through this connection, which steers the agent away from expecting a delete/undo path.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_create_milestoneCreate MilestoneAInspect

Create a milestone in a project (status: Open), optionally with description, start/end dates, tags, and existing runs attached.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoLabels
ownerNoMilestone owner: an organization member (email, username, or user id). Omit for no owner.
titleYesMilestone title
end_atNoEnd date (YYYY-MM-DD)
start_atNoStart date (YYYY-MM-DD)
project_idYesProject ID (ULID)
descriptionNoWhat this milestone covers
test_run_idsNoExisting run ids to attach

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false, and openWorldHint=false, so the write-but-non-destructive profile is covered. The description adds one genuinely useful behavioral detail — the created milestone defaults to status 'Open' — but says nothing about permissions, whether attached runs must belong to the project, or what the call returns.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single front-loaded sentence stating the core action followed by the optional surface area; no filler and nothing redundant. Appropriate size for a create tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With 8 well-documented parameters, required fields, and annotations covering the safety profile, the description covers enough for correct invocation. It only omits return/ID behavior, which is a minor gap given there is no output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (owner, dates, tags, test_run_ids all documented in the schema), so the description's paraphrase of those fields adds little. Baseline 3 is appropriate when the schema carries the parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb+resource ('Create a milestone in a project') and enumerates the optional payload (description, dates, tags, runs), so it is easily distinguished from siblings like tm_update_milestone and tm_create_project. An agent can identify what the tool produces without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied by the verb 'Create'; there is no explicit when-to-use guidance, no note about prerequisites (e.g., project must exist), and no routing to alternatives such as tm_update_milestone for modifying an existing milestone. Adequate but with clear gaps.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_create_projectCreate ProjectAInspect

Create a new Test Manager project. Project & Org Instructions (Memory Layer) stay managed in the UI. Deleting projects is not available through this connection.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesProject name
tagsNoLabels
descriptionNoWhat the project covers

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds genuinely non-obvious behavioral context: instructions live in the UI and deletion is unavailable through this connection, which prevents an agent from assuming a full CRUD surface.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with the core action, then two tightly-scoped caveats. No filler or restated title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-required-param create tool with annotations covering safety and no output schema, the essentials are present. A minor gap is that it doesn't hint at the result (e.g., whether a project ID is returned), which would matter for chaining into tm_create_folder or tm_create_test_case.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 100% schema description coverage and only three simple params (name, tags, description), the schema carries the parameter meaning. The description adds no format, constraint, or default details beyond it, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Create a new Test Manager project"), which clearly separates it from sibling creation tools like tm_create_folder or tm_create_test_case. It stops short of explicitly contrasting those siblings, but the resource scoping is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The note that "Project & Org Instructions (Memory Layer) stay managed in the UI" implicitly tells the agent not to attempt that here, and the deletion note rules out one adjacent action. However, there is no explicit when-to-use guidance or routing to alternatives such as tm_update_project or tm_create_folder.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_create_test_caseCreate Test CaseAInspect

Create a single test case in a project folder, with optional steps and references to existing step Modules (module steps are included; the Module itself is unchanged). For creating many cases from a requirement, prefer tm_generate_test_cases.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoLabels
typeNoe.g. Functional, Regression, Smoke & Sanity
titleYesTest case title
statusNoe.g. Live, Unverified, Ready (default per project settings)
priorityNoLowest, Low, Medium, High or Highest
folder_idNoFolder to file the case into
module_idsNoExisting step Module ids to include (appended as module steps after test_steps)
project_idYesProject ID (ULID)
test_stepsNoOrdered steps
descriptionNoWhat the case verifies
preconditionsNoSetup required before the steps
estimated_timeNoEstimated execution time in minutes
automation_statusNoAutomated, Not Automated or To Be Automated

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safety profile (readOnlyHint=false, destructiveHint=false, openWorldHint=false), so the bar is lower. The description still adds real behavioral context: module steps are included and appended after test_steps, and critically that 'the Module itself is unchanged' — a non-obvious side-effect clarification the annotations do not convey. It stops short of disclosing what is returned or any permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tightly written sentences, front-loaded with the create semantics before the routing hint. No filler and no repetition of the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 13-parameter mutation tool with no output schema, the description covers create semantics and the bulk alternative but says nothing about the response (e.g., the created case's id) which the agent will need for follow-up calls. Folder placement when folder_id is omitted and the project-default status behavior are left entirely to the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already documents all 13 parameters, including the module append-order note and the 'outcome' key constraint. The description adds almost nothing beyond the schema, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a precise verb (Create), resource (a single test case), and scope (in a project folder, with optional steps and module references), and explicitly contrasts itself with the bulk sibling tm_generate_test_cases. An agent can route between the two without opening either schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly names the condition that selects the alternative: 'For creating many cases from a requirement, prefer tm_generate_test_cases.' That is a clear when-to-use-this vs. when-to-use-something-else statement, plus an implicit boundary (single vs. many).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_create_test_runCreate Test RunAInspect

Create a test run from chosen test cases — manual (default) or KaneAI (run_type: "kaneai"). Each included test case becomes a TEST RUN INSTANCE — the executable copy of that test inside the run, which carries its own status, remarks and exactly one Configuration. The cases must match the run type: KaneAI runs only take KaneAI-authored cases with completed code generation; manual runs only take manual cases. Optionally attach existing milestones, file into a run folder, and apply a Configuration to all instances (KaneAI configurations must be compatible with each case's platform — web/mobile-app/mobile-browser and iOS/Android).

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoLabels
titleYesRun title
assigneeNoAssign every created instance to this organization member: email, username, or user id. Omit to leave the instances unassigned.
run_typeNoRun type (default manual). "kaneai" creates a KaneAI run: only KaneAI-authored test cases can be included, and configurations must be kane-supported and platform-compatible.
folder_idNoTest-run folder to file the run into
objectiveNoWhat this run verifies
project_idYesProject ID (ULID)
is_sequentialNoKaneAI runs only: execute the instances sequentially (default false).
milestone_idsNoExisting milestone ids to attach the run to
test_case_idsNoTest case ids to include
configuration_idNoApply ONE Configuration to every included test case (the configuration's top-level id from tm_list_configurations)
configuration_idsNoApply MULTIPLE Configurations: each included test case gets one test run instance per configuration (case × configuration), like the UI's Add Configuration checkboxes

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false, openWorldHint=false, so the safety profile is covered. The description adds real behavioral context beyond that: each case is copied into an executable instance with its own status/remarks/one Configuration, and it exposes a precondition failure mode (mismatched run type, incomplete code generation, incompatible platform).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core action and mode choice, and the TEST RUN INSTANCE definition earns its space because it clarifies the object model. The final sentence packs several constraints (milestones, folder, configuration, platform compatibility) into one dense clause, slightly hurting scanability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 12-parameter create tool with full schema coverage, the description covers run types, prerequisites, instance semantics, and configuration behavior well. With no output schema present, it stops short of describing what the call returns (e.g., the created run/instances), which is the remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description still adds value by explaining the configuration_id vs configuration_ids distinction (one Configuration shared by all instances vs case×configuration expansion) and the KaneAI platform-compatibility rule, neither of which is fully stated in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb+resource ('Create a test run') with the two run modes and their semantic difference spelled out, plus a definition of what an included case becomes (a TEST RUN INSTANCE). No other sibling tool creates a run, so the agent can disambiguate from tm_create_test_case, tm_create_milestone, and tm_generate_test_cases immediately.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context and prerequisites: KaneAI runs only accept KaneAI-authored cases with completed code generation, manual runs only accept manual cases, and options are described as optional (milestones, folder, configuration). It never routes to a named alternative (e.g., when to use tm_generate_test_cases first), so it stops short of explicit when-not guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_generate_test_casesGenerate Test Cases with AIAInspect

Generate structured test cases from a requirement or prompt using the Test Manager AI generator (applies the project's Memory Layer — Project & Org Instructions — automatically) and file them into the chosen folder. Requirement documents, specs, spreadsheets or screenshots can be attached via files to ground the generation. Volume = test_scenario_limit × per_scenario_test_cases_limit, capped at 50 per call. By default all generated cases are saved into the folder (one sub-folder per scenario); pass auto_save=false to review before saving. The call returns FAST with a request_id (and a browser progress URL) — generation takes ~30-90s in the background; call this tool again with just request_id to check progress, fetch results, or save. Note: consumes AI generation credits.

ParametersJSON Schema
NameRequiredDescriptionDefault
filesNoOptional reference files to ground the generation (requirement docs, specs, data sheets, screenshots). Each item carries the file name and its raw bytes base64-encoded in content_base64. Supported: .pdf, .docx, .txt, .csv, .xlsx, .json, .xml, .jpg, .jpeg, .png, .gif, .bmp, .webp; max 5 files, 10 MB each. Large files are better uploaded in the Test Manager UI, since the base64 content travels in the request. Only used when starting a new generation.
promptNoThe requirement / instructions to generate test cases from. Required when starting a new generation.
jira_idsNoOptional Jira issue keys to use as requirement context
auto_saveNoSave all generated cases into the folder automatically (default true for a NEW generation). Set false to review first — that choice persists across request_id polls (a bare poll never saves it); to save after review, call again with request_id AND auto_save=true.
folder_idNoFolder to file generated cases into. Required when starting a new generation.
project_idNoProject ID (ULID). Required when starting a new generation.
request_idNoResume/act on an earlier generation: check progress, fetch results, or save them.
test_scenario_limitNoMax scenarios to generate (default 3)
per_scenario_test_cases_limitNoMax test cases per scenario (default 3)

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=false and destructiveHint=false; the description adds the async contract (returns FAST with request_id + progress URL, ~30-90s background), the credit consumption, automatic Memory Layer application, and the 50-case cap. These are behavioral traits the agent cannot get from structured fields.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The purpose and key constraints (files, volume cap, auto_save default, async flow, credits) are front-loaded and nearly every sentence carries actionable detail. It is dense but not padded, though the parenthetical Memory Layer note slightly interrupts flow.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For an async, no-output-schema, 9-parameter generation tool, the description covers the return value (request_id + progress URL), the two-phase workflow, saving behavior, and cost — everything an agent needs to invoke and follow up correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so parameters are already documented, but the description adds real semantics: the volume formula (test_scenario_limit × per_scenario_test_cases_limit, cap 50), and the request_id resume/poll behavior including that the auto_save=false choice persists across polls.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('Generate structured test cases from a requirement') and names the generator and destination folder. It is clearly distinguishable from the manual sibling tm_create_test_case by emphasizing AI generation from a prompt, not hand-authored creation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly covers when to use which mode: attach files to ground generation, pass auto_save=false to review before saving, and re-call with just request_id to check progress/fetch/save. It also warns that generation consumes AI credits, which is a genuine when-to-use consideration.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_get_coverage_summaryGet Coverage SummaryA
Read-only
Inspect

Test coverage for a project, overall and per folder, computed live from the project's test cases (matches the Test Cases screen). Also answers requirement and traceability coverage: pass jira_id (any Jira issue key, e.g. TE-123: a requirement, user story, epic, ticket or bug) to get that item's test coverage (the test cases linked to it and how many are automated; zero linked cases means uncovered). For a set of requirements, call once per Jira key. A scope with no test cases returns all zeros; a Jira key that does not exist in the connected Jira returns an error.

ParametersJSON Schema
NameRequiredDescriptionDefault
jira_idNoOptional Jira issue key (e.g. TE-123) — scope coverage to the test cases linked to this requirement / user story / epic / ticket (traceability coverage)
folder_idNoOptional folder scope
project_idYesProject ID (ULID)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, openWorldHint=false, so the safety profile is covered. The description adds genuine behavioral context beyond that: coverage is computed live from test cases, a scope with no test cases returns all zeros, and a nonexistent Jira key returns an error rather than an empty result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single dense paragraph with the core purpose front-loaded, followed by the jira_id mode and edge-case behavior. Every sentence carries information, though the middle clause enumerating Jira issue types is slightly over-packed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description correctly compensates by describing what comes back (per-folder and overall coverage, zeros for empty scope, error for unknown Jira keys) and how automation counts factor in. It stops short of describing the actual response shape or field names.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3, but the description adds real semantics the schema doesn't carry: jira_id accepts any Jira issue type (requirement, user story, epic, ticket, bug) and zero linked cases explicitly means uncovered. folder_id and project_id get no extra detail, so it does not reach 5.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource (test coverage for a project, overall and per folder) plus a second mode (requirement/traceability coverage via jira_id), and anchors it to the Test Cases screen. No sibling tool computes coverage, so the distinction is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit routing: pass jira_id for requirement/traceability coverage, and 'For a set of requirements, call once per Jira key' tells the agent how to batch. It lacks an explicit 'when not to use this' or named sibling alternative, keeping it short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_get_milestoneGet MilestonesA
Read-only
Inspect

Pass milestone_id to get one milestone's detail and live completion progress. Otherwise pass project_id to list the project's milestones.

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idNoProject ID (ULID). Required when milestone_id is not given.
milestone_idNoMilestone ID (ULID). When given, returns detail + progress.

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and closed-world scope, so the safety profile is covered. The description adds only the note that detail mode returns 'live completion progress', which is mild extra context. No mention of ordering, pagination, or empty-result behavior for the list mode.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no filler, and the distinguishing condition (milestone_id vs project_id) is front-loaded in the first clause. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-parameter read-only tool with full schema coverage and safety annotations, the description covers the essentials an agent needs to choose a call path. It is slightly thin on list-mode behavior, but nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters already document ULID format, requiredness, and conditional behavior. The description restates the same either/or logic rather than adding new meaning, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (get milestone detail vs list milestones) and clearly separates the two modes by which parameter is supplied. It does not name or differentiate against any sibling tool, but the dual-mode wording is unambiguous about what comes back.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives an explicit conditional: pass milestone_id for one milestone, otherwise pass project_id to list. That is clear routing logic for a single tool. It stops short of naming alternatives (e.g., tm_list_projects) or stating exclusions, so it does not reach 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_get_test_casesFind / Get Test CasesA
Read-only
Inspect

Find test cases in a project by filters (folder, priority, automation status, type, tags) and/or search (matches test case ID like TC-123, or title; minimum 3 characters) — returns summaries. Filters combine (AND). Pass case_id instead to get one case's full detail (steps, BDD scenarios, field values).

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-based)
tagsNoFilter by tags
typeNoFilter by test type (e.g. Functional, Regression, Smoke & Sanity)
limitNoMax results per page
searchNoFull-text keyword (min 3 chars)
case_idNoTest case ID (ULID). When given, returns that case's full detail and all other filters are ignored.
priorityNoFilter by priority (e.g. Lowest, Low, Medium, High, Highest). A value outside the project's priority list is rejected with the valid values.
folder_idNoOnly cases in this folder
project_idNoProject ID (ULID). Required unless case_id is given.
automation_statusNoFilter by automation status (e.g. Automated, Not Automated, To Be Automated)

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, so the safety profile is covered. The description still adds real behavioral context: results are summaries, the search must be at least 3 characters, filters combine via AND, and case_id silently overrides all other filters. That override behavior is a genuinely useful warning the annotations do not convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two compact sentences, front-loaded with the primary search use case before the single-case alternative. Every clause carries information; no filler or restatement of the title.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description partly compensates by naming the return shape (summaries vs. steps/BDD scenarios/field values). It omits any mention of pagination behavior despite page/limit parameters, which is the main remaining gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by clarifying what `search` matches (test case ID like TC-123 or title), the AND semantics of filters, and that case_id causes other filters to be ignored — semantics the schema only partially states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names the specific resource (test cases in a project) and the two distinct modes: filtered search returning summaries, and single-case lookup via case_id returning full detail. That distinction lets an agent tell this apart from the sibling list/search tools without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives explicit routing within the tool ('Pass case_id instead to get one case's full detail') and states that filters combine with AND. It does not name a sibling tool (e.g. tm_get_test_runs, tm_list_folders) as an alternative, so it stops short of the 5 rubric's explicit alternative naming.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_get_test_runsGet Test RunsA
Read-only
Inspect

Pass run_id to get one test run's detail plus its result summary (instance counts by status — passed / failed / skipped / not started / in progress — and pass rate). Otherwise pass project_id to list runs with optional filters. Every returned run carries run_type ("kaneai" or "manual") identifying KaneAI runs — the backend stores KaneAI runs with type "Manual" (result-entry mode), so use run_type, not type, to tell them apart.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-based)
limitNoMax results per page
run_idNoTest run ID (ULID). When given, returns detail + result summary.
searchNoSearch runs by title
statusNoFilter: Not Started, In Progress, Passed, Failed, Skipped
run_typeNoFilter: "kaneai" (KaneAI runs), "manual" (human result-entry runs, KaneAI excluded), or "automation" (framework-uploaded runs). Buckets are disjoint.
folder_idNoOnly runs in this folder
project_idNoProject ID (ULID). Required when run_id is not given.

TDQS

A4.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so safety is covered. The description does add genuinely non-obvious context about the backend storing KaneAI runs as type "Manual" and instructing use of run_type instead of type, which is valuable. It stops short of covering pagination behavior for the list mode.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three dense sentences with no filler. The primary routing rule is front-loaded, and the caveat about run_type is placed last where it matters most for interpretation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description shoulders the return-value burden and does describe the detail payload (instance counts by status, pass rate). Pagination semantics for the list branch (page/limit) are left to the schema, a minor gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description goes beyond the schema by clarifying the run_id-vs-project_id dependency and by warning that run_type, not the backend's type field, is the reliable discriminator — real added meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource and distinguishes the two operating modes: run_id yields a single run's detail plus result summary, while project_id lists runs with filters. An agent knows exactly what it will get without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes usage: pass run_id for one run, otherwise pass project_id to list. The alternative branch and the selecting condition are both stated, so no inference is required.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_list_configurationsList Run ConfigurationsA
Read-only
Inspect

List saved run Configurations (reusable platform / browser / OS / device setups). To apply one to a run, pass the configuration's TOP-LEVEL id to tm_create_test_run or tm_update_test_run as configuration_id (NOT the nested environments[].environment_id). Pass run_type to see only the configurations usable for that run type (kaneai vs manual). Each test instance carries exactly one configuration. Configurations are created by humans in the Test Manager UI. They are organization-level (shared by every project), so there is no project filter. Results are paged: the response says how many exist in total; page through them or raise limit to see all.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-based)
limitNoMax results per page (default 10)
run_typeNoOnly configurations supporting this run type (kane-supported vs manual-supported)

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false and openWorldHint=false, and the description adds substantial context beyond that: configurations are human-created in the UI (not via API), are organization-level and shared across all projects, and results are paged with a total count in the response. These are non-obvious behavioral traits an agent needs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core purpose, then dense, high-value sentences on the id distinction and paging. Slightly long, but nearly every sentence carries actionable information; the 'each test instance carries exactly one configuration' line is marginally less essential.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description still covers return behavior (total count, pagination) and the org-level scope constraint. Combined with annotations covering the safety profile, an agent has everything needed to call this correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds real semantic value: it warns that the TOP-LEVEL id (not nested environments[].environment_id) must be used as configuration_id, and clarifies run_type's kaneai vs manual meaning. Page/limit are only referenced generically ('page through them or raise limit').

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (List) and resource (saved run Configurations) and defines what a configuration is (reusable platform/browser/OS/device setups). It also distinguishes itself from the sibling tools tm_create_test_run and tm_update_test_run by explaining that this tool is the source of the configuration_id those tools consume.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context: pass run_type to narrow by kaneai vs manual, and notes there is no project filter because configurations are org-level. It routes the agent to sibling tools for applying a configuration but doesn't state explicit when-not-to-use conditions beyond the implicit 'use tm_create_test_run to apply'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_list_foldersList FoldersA
Read-only
Inspect

Get a folder tree. Test Manager keeps three folder types: Test Cases and Test Runs folders (per project — pass project_id) and Modules folders (organization-level, since modules are shared across projects — no project_id needed). Pick with the tree parameter (default: test-cases). New test-case/test-run folders can be created with tm_create_folder.

ParametersJSON Schema
NameRequiredDescriptionDefault
treeNoWhich folder tree to return (default test-cases). "modules" is organization-level.
project_idNoProject ID (ULID) — required for the test-cases and test-runs trees; not used for modules

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint/openWorldHint/destructiveHint, so safety is covered. The description adds real data-model behavior beyond that: modules are organization-level and shared across projects while test-case/test-run folders are project-scoped. It does not describe return shape or pagination, but no output schema exists to carry that either.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tightly written sentences with the core action front-loaded, followed by the type taxonomy and the parameter rule. No filler and nothing repeated that the schema already says at greater length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a two-param read tool with full schema coverage and safety annotations, the description supplies everything needed to choose the right tree and parameters. The only gap is return format/pagination, which is minor given there is no output schema to align with.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so both parameters and the enum are already documented in the schema, making this the baseline case. The description restates the tree default and the tree/project_id coupling, which is mildly reinforcing but largely duplicative of the schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States the specific verb and resource ('Get a folder tree') and immediately enumerates the three folder types the Test Manager model contains, including which are per-project vs organization-level. An agent can distinguish this from tm_list_modules and tm_list_projects without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear context: pass project_id for test-cases/test-runs, omit it for modules, and select via the tree parameter (default test-cases). It also routes creation work to tm_create_folder. It stops short of an explicit 'do not use this for X' exclusion, but coverage is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_list_modulesList Step ModulesA
Read-only
Inspect

List the organization's reusable step Modules (modules are org-level, shared across projects — the same list as the Modules page). Supports name search and pagination. Reference a module by id when creating or editing a test case to include its steps (the Module itself is never changed).

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-based)
limitNoMax results per page (default 100)
searchNoFilter modules by name keyword
folder_idNoOnly modules in this Modules folder (folder ids from tm_list_folders with tree="modules")
include_subfoldersNoWith folder_id: also include modules from its subfolders

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly/non-destructive/non-open-world, so safety is covered. The description adds genuinely useful context beyond that: modules are org-level and shared, the list mirrors the Modules page, pagination is supported, and referencing a module never mutates it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences with purpose front-loaded and the cross-reference to test-case creation placed last. Slightly padded by the parenthetical restating the Modules-page equivalence, but nothing is confusing or out of order.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description conveys enough for correct invocation: read-only listing, org scope, search/pagination support, and how ids are consumed. Minor gaps (return shape, default page size) are covered by the schema's parameter docs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all five parameters (page, limit, search, folder_id, include_subfolders) are already documented. The description only mentions search and pagination generically, adding no syntax or semantics beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Specific verb + resource ('List the organization's reusable step Modules') with scope defined (org-level, shared across projects, equivalent to the Modules page). No sibling tool lists modules, and the description makes clear this is distinct from folders/projects/test cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives a concrete downstream use case: reference a module by id when creating or editing a test case. It does not state when NOT to use it or name an alternative, but there is no competing module-listing sibling, so the guidance is clear if not exhaustive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_list_projectsList Test Manager ProjectsB
Read-only
Inspect

List all Test Manager projects the authenticated user is allowed to see. Supports keyword search and pagination.

ParametersJSON Schema
NameRequiredDescriptionDefault
pageNoPage number (1-based)
limitNoMax results per page (default 25)
searchNoFilter projects by name keyword

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, destructiveHint=false, and openWorldHint=false, so the safety profile is covered. The description adds the meaningful access-control nuance that results are limited to what the user is permitted to see. It says nothing about ordering, pagination limits, or result shape beyond what the schema states.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences that front-load the core scope before the optional capabilities. No filler or repetition; only slightly short of ideal since the second sentence is a bare feature list.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple read-only list tool with full schema coverage and clear annotations, and no output schema is needed. The description covers scope, filtering, and pagination adequately; only guidance about defaults and result ordering is absent, which is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so page, limit, and search are fully documented in the schema. The description's mention of keyword search and pagination merely restates those parameters without adding format or edge-case detail, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('List all Test Manager projects') plus an authorization scope ('the authenticated user is allowed to see'). An agent can distinguish it from write-oriented siblings like tm_create_project or tm_update_project, though it is not differentiated from other tm_list_* readers such as tm_list_folders or tm_list_modules.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description identifies supported capabilities ('keyword search and pagination') but gives no guidance on when to choose this tool over alternatives such as tm_get_test_runs or tm_list_folders, nor any prerequisites or exclusions. Usage is only inferable from the tool's name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_record_test_resultsRecord Test Results (Bulk)A
Destructive
Inspect

Record execution results on a MANUAL run's TEST RUN INSTANCES (the per-run copies of each test case) — max 500 per request, all-or-nothing (larger requests are rejected with nothing recorded). Manual runs only: KaneAI run results are recorded by KaneAI executions, so manual entry on a KaneAI run is rejected. Target an instance by instance_id, or just pass test_case_id and the instance is resolved automatically. Valid statuses: Passed, Failed, Skipped, Not Started (terminal — run-level "In Progress" is set via tm_update_test_run). Re-recording a step status replaces its result: an omitted step remark is cleared; response echoes per-step outcomes. Optional remarks and assignee per result, and per-step results via steps[] (mark individual test steps / BDD scenario rows Passed/Failed/Skipped with remarks, like the UI's per-step Mark Status). Note: this sets INSTANCE statuses; the run's overall status is set separately via tm_update_test_run.

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesTest run ID (ULID)
resultsYesResults to record (max 500)

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the destructiveHint=true / readOnlyHint=false annotations: max 500 per request, all-or-nothing rejection, terminal-only instance statuses, and the non-obvious replacement behavior (re-recording a step clears an omitted remark, response echoes per-step outcomes). This is exactly the operational detail an agent needs before mutating results.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads the core purpose and is dense with useful content, but the run-vs-instance status note is stated twice (once parenthetically in the status list and again as a closing 'Note'), which is mild redundancy in an otherwise tightly packed paragraph.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, yet the description still tells the agent the response echoes per-step outcomes. Combined with batch limits, failure semantics, status validity, and the run-level hand-off, nothing an agent needs to invoke this mutating tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine resolution semantics: instance_id takes precedence while test_case_id triggers automatic resolution, plus the valid status set and the all-or-nothing constraint on the results array. It stops short of detailing remarks/assignee formats, which the schema already carries.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource — recording execution results on test run instances — and immediately narrows scope to MANUAL runs' per-run copies. It also distinguishes itself from the run-level sibling (tm_update_test_run) and from KaneAI executions, so an agent can tell it apart without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when the tool applies (manual runs, targeting by instance_id or auto-resolved test_case_id) and when it does not ('KaneAI run results are recorded by KaneAI executions, so manual entry on a KaneAI run is rejected'). It names the alternative for run-level status changes, tm_update_test_run, and clarifies that this tool only sets instance statuses.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_update_folderUpdate FolderA
Destructive
Inspect

Rename a folder, change its description, or move it under a different parent folder. Works on all three folder types: per-project Test Cases (default) and Test Runs trees, and the organization-level Modules tree. Moves of test-cases/test-runs folders require project_id; module folders need none.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name
treeNoWhich folder tree the folder belongs to (default test-cases). "modules" is organization-level.
folder_idYesFolder ID (ULID)
project_idNoProject ID (ULID) — required when moving a test-cases or test-runs folder; not used for modules
descriptionNoNew description
parent_folder_idNoMove under this folder (ULID), or pass "root" to make it a top-level folder; cannot be the folder itself or one of its descendants

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true and readOnlyHint=false, so the mutation risk is carried by structured data. The description usefully adds that moves have tree-dependent project_id requirements, but says nothing beyond the schema about reversibility or what happens to existing children.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the three supported operations before the tree-scope constraints. Every clause earns its place with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool whose safety profile and parent constraints are covered by annotations and schema, the description supplies the critical tree-type/project_id conditional. Minor gaps like uniqueness or failure behavior remain but are not essential to invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3, but the description adds genuine semantic value by clarifying the conditional relationship between tree type and project_id that the flat schema cannot express. It reinforces rather than merely repeats the parameter docs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States specific verbs and resource: rename, change description, move, on folders. Distinguishes the three folder-tree scopes (Test Cases, Test Runs, Modules) so the agent understands the tool's reach versus siblings like tm_create_folder or tm_list_folders.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains the conditional context: test-cases/test-runs moves require project_id while module folders need none, and 'modules' is organization-level. Clear usage context, though it doesn't explicitly name an alternative tool for editing non-folder entities.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_update_milestoneUpdate MilestoneA
Destructive
Inspect

Update a milestone — title, description, dates, tags — or change its status (e.g. mark it Completed). Deleting milestones is not available through this connection.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoReplace tags
ownerNoChange the milestone Owner — an organization member (email, username, or user id).
titleNoNew title
end_atNoEnd date (YYYY-MM-DD)
statusNoNew status: Open (reopen) or Complete — the UI's two milestone states. "Active" and "Completed" are accepted as aliases and normalized to Open/Complete (any other stored value hides the milestone from the UI lists).
start_atNoStart date (YYYY-MM-DD)
project_idYesProject ID (ULID)
descriptionNoNew description
milestone_idYesMilestone ID (ULID)

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false and destructiveHint=true, so the write/overwrite risk profile is covered. The description adds a genuine constraint (no deletion path) but omits update semantics that matter for a destructive tool — whether omitted fields are left unchanged, whether tags replace or append, and any permission requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, capability first and the exclusion second, both front-loaded with no filler. There is slight redundancy in re-listing fields that the schema already enumerates, which keeps it just short of ideal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 9-parameter mutation tool with full schema coverage, no output schema, and annotations covering the safety profile, the description is nearly sufficient. The missing piece is partial-update semantics and required-permission context, which an agent would want before issuing a destructiveHint=true call.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and includes richer detail than the description (status alias normalization to Open/Complete, ULID formats, tags replacement, owner resolution). The description only restates the field list, adding no parameter meaning beyond the schema, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Update) plus resource (milestone) and enumerates the mutable surface — title, description, dates, tags, status. It does not name or contrast with sibling tools (tm_get_milestone, tm_create_milestone, tm_update_project), so an agent must rely on the tool name alone to disambiguate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives one useful boundary — 'Deleting milestones is not available through this connection' — which rules out an adjacent intent, and the status example implies a reopen/complete workflow. However, there is no guidance on when to prefer this over tm_update_project or how to discover the milestone_id (via tm_list_projects/tm_get_milestone) first.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_update_projectUpdate ProjectA
Destructive
Inspect

Update a project's name, description or tags (fields are merged — only what you pass changes). Project & Org Instructions stay managed in the UI.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name
tagsNoReplace tags
project_idYesProject ID (ULID)
descriptionNoNew description

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare destructiveHint=true and readOnlyHint=false. The description adds valuable behavioral context: patch-style merge semantics and that instructions cannot be updated here. It does not cover permissions, error behavior, or response shape, but those gaps are less critical given the safety annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two tight sentences with no filler. It front-loads the core action and then immediately adds the key merge and restriction details, both of which earn their place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple update tool with full schema coverage, safety annotations, and no output schema, the description covers the essential scope and merge behavior. It could mention that project_id is required or describe error cases, but those are minor given the structured fields already available.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameter meaning is already fully documented. The description restates the updatable fields and adds merge semantics, but it does not add param-specific syntax or constraints beyond what the schema provides. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb and resource: update a project's name, description or tags. This clearly distinguishes it from siblings like tm_create_project and tm_update_folder. It also signals the update scope, so an agent can identify the tool without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It states that fields are merged and only passed fields change, which tells the agent how to invoke it. It also clarifies that Project & Org Instructions remain UI-managed, giving a useful when-not boundary. However, it does not explicitly name alternative tools or elaborate on when to choose this over other update operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_update_test_caseUpdate Test CaseA
Destructive
Inspect

Edit an existing test case (fields are merged — only what you pass changes). test_steps, when given, REPLACE the case's steps; module_ids append module reference steps. Deleting test cases is not available through this connection.

ParametersJSON Schema
NameRequiredDescriptionDefault
tagsNoReplace tags
typeNoTest type
titleNoNew title
statusNoe.g. Live, Archived, Faulty, Ready
case_idYesTest case ID (ULID)
priorityNoLowest, Low, Medium, High or Highest
folder_idNoMove to this folder
module_idsNoExisting step Module ids to append as module steps
project_idYesProject ID (ULID)
test_stepsNoReplacement steps (omit to keep current steps)
descriptionNoNew description
preconditionsNoNew preconditions
estimated_timeNoEstimated minutes
automation_statusNoAutomated, Not Automated or To Be Automated

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint=true and readOnlyHint=false; the description earns credit by specifying exactly what is destroyed ('test_steps, when given, REPLACE the case's steps') and by disclosing the merge invariant for all other fields. This is precisely the additive context the destructive hint cannot convey.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three tight sentences with zero filler; the merge rule, the replace/append distinction, and the deletion exclusion are all front-loaded before any field-specific detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 14-parameter mutation tool with no output schema, it covers the highest-risk behaviors (partial update, step replacement, no delete). It omits permission/auth requirements and whether module_ids appends interact with test_steps replacement, which would be the next most useful detail.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description goes slightly beyond by stating a tool-wide merge rule that governs every optional parameter, and by contrasting replace (test_steps) with append (module_ids) semantics that would otherwise have to be inferred field-by-field.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('Edit an existing test case') and immediately qualifies the semantics with 'fields are merged — only what you pass changes'. It implicitly contrasts with tm_create_test_case/tm_update_test_run by saying 'existing', though it never names a sibling explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives clear operational guidance: partial updates are safe (merge), test_steps replaces rather than appends, module_ids appends, and deletion is explicitly unavailable 'through this connection'. It rules out the delete use case but does not name the alternative tool or surface for deletion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

tm_update_test_runUpdate Test RunA
Destructive
Inspect

Update an open test run — overall status (Passed / Failed / Skipped / In Progress), title, objective, milestones, folder, add test cases (each added case becomes a new test run instance; cases must match the run's type — KaneAI or manual), apply a Configuration to the run's instances (configuration_id; on KaneAI runs it must be kane-supported and platform-compatible with the targeted cases), toggle sequential execution (KaneAI runs), or archive it. Only open (active) runs can be updated; a closed/archived run returns a clear message. Adding cases/milestones appends, never replaces.

ParametersJSON Schema
NameRequiredDescriptionDefault
titleNoNew title
run_idYesTest run ID (ULID)
statusNoSet the run's OVERALL status (independent of per-instance results). Manual runs only — KaneAI run status is managed by KaneAI executions.
archiveNoArchive (close) the run
assigneeNoAssign the run's instances to this organization member: email, username, or user id ("none" clears). Scope with apply_to_test_case_ids. Omit to keep the current assignee.
folder_idNoMove to this run folder
objectiveNoNew objective
is_sequentialNoKaneAI runs only: execute the instances sequentially.
milestone_idsNoMilestone ids to attach (appended)
configuration_idNoSet ONE Configuration on the run's instances (the configuration's top-level id from tm_list_configurations; each instance carries exactly one configuration)
add_test_case_idsNoTest case ids to add to the run (appended)
configuration_idsNoAPPEND multiple Configurations: each targeted test case gets an additional instance per configuration (case × configuration, like the UI); existing configured instances are untouched
apply_to_test_case_idsNoWith configuration_id or configuration_ids: only apply to these test cases (default: all instances in the run)

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations declare destructiveHint=true but not what is destroyed; the description discloses that archive closes the run and, importantly, that adds are append-only ('Adding cases/milestones appends, never replaces') and that each added case spawns a new run instance. It does not mention permissions/auth or reversibility of archive. No contradiction with the readOnlyHint=false / destructiveHint=true annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core action, then a dense but organized enumeration of mutatable fields with their constraints. It is long for one paragraph and relies on semicolons, but for a 13-parameter tool with several interacting modes almost every clause carries operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, yet the description covers the exception path ('a closed/archived run returns a clear message') and the append-vs-replace behavior an agent must anticipate. Auth requirements and the exact behavior of configuration_ids with apply_to_test_case_ids defaults could be clearer, but the definition is otherwise self-sufficient for a complex update tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description adds cross-parameter meaning the schema alone does not: configuration_id replaces vs configuration_ids appends, apply_to_test_case_ids scopes both, and configuration_id must be kane-supported and platform-compatible on KaneAI runs. That is real semantic value beyond the per-field text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Update an open test run') and then enumerates the exact facets that can be changed (status, title, objective, milestones, folder, cases, configuration, sequential, archive). An agent can immediately separate this from tm_create_test_run, tm_get_test_runs and tm_update_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete preconditions: 'Only open (active) runs can be updated; a closed/archived run returns a clear message', plus field-level gating (status is manual-only, is_sequential is KaneAI-only, cases must match the run's type). It stops short of naming a sibling alternative for related write paths (e.g. tm_record_test_results), but the when-conditions are explicit.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

triggerSuiteRunTrigger Suite RunAInspect

Triggers a phone call test run for a specific suite. This will initiate phone calls for all scenarios configured in the suite.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesThe project ID that contains the suite
suiteNameYesThe name of the suite to trigger

Output Schema

ParametersJSON Schema
NameRequiredDescription
dataNoCall initiation results
messageNoStatus message
successNoWhether the call initiation was successful

TDQS

A3.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, openWorldHint=true, destructiveHint=false, and the description adds genuinely new behavioral context: this initiates real outbound phone calls for every configured scenario, a costly physical-world side effect. It still omits async behavior, whether the run can be cancelled/stopped, and whether it is idempotent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, zero filler, with the action and its blast radius ('all scenarios') front-loaded. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a trigger tool with a 2-parameter fully documented schema and an output schema that covers returns, the description is nearly sufficient. The remaining gap is operational: it does not say how to track the launched run or whether the call is asynchronous, which matters for a long-running phone-call test.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with both required parameters (projectId, suiteName) fully documented in the schema itself. The description adds no syntax, format, or naming-convention detail beyond what the schema already provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb plus resource ('Triggers a phone call test run for a specific suite') and clarifies scope by noting it fires every scenario in the suite. No sibling is named, but no other tool in the list triggers a suite run, so confusion risk is low.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains the effect but never says when to use it versus alternatives (e.g. getSuiteOverview for inspecting a suite, or how to observe the resulting run). No prerequisites, permissions, or preconditions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateBuildRename Automation BuildB
DestructiveIdempotent
Inspect

Renames a LambdaTest web automation build. Overwrites the current build name.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesNew build name
buildIdYesID of the build to rename

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare destructiveHint=true, idempotentHint=true, and readOnlyHint=false, so the safety profile is covered. The description adds the useful detail that the existing name is overwritten (the old value is lost), but says nothing about permissions, error cases, or whether buildId must reference an existing build.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the action and followed by the one behavioral caveat. Nothing is wasted and nothing is buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter mutation with full schema coverage and annotations declaring destructiveness and idempotency, the description is nearly sufficient. It lacks only error/permission context, which is minor here.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both 'name' and 'buildId' documented inline, so the schema carries the burden. The description adds no format, constraint, or lookup detail beyond that, which is the expected baseline when the schema is complete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Renames a LambdaTest web automation build.' An agent can distinguish this from deleteBuild, stopBuild, and updateSession without inspecting schemas, though it does not explicitly name which sibling to prefer.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance, no prerequisites, and no mention of alternatives such as deleteBuild or updateSession. The agent must infer the trigger context entirely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

updateSessionUpdate SessionA
Destructive
Inspect

Update the name or pass/fail status of a test session. Use status_ind to mark tests as passed, failed, error, or skipped from your test framework.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoNew name for the session
sessionIdYesThe session ID to update
status_indNoTest result status

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false and destructiveHint=true, so the safety profile is covered structurally. The description adds the status_ind semantics but says nothing about what the update overwrites, reversibility, or auth requirements, so it contributes only marginal context beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two tight sentences that front-load the resource and the two updatable fields, with no wasted verbiage. Slightly terse rather than maximally informative, but well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a low-complexity, three-parameter mutation with full schema coverage and annotations covering safety, the description is nearly sufficient. The only gap is the absence of any statement about effect of the change or the session ID requirement, which the schema partially covers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions on all three parameters, so the schema already carries the parameter documentation. The description's note that status_ind comes from the test framework lightly reinforces the enum but adds no syntax or format detail beyond the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (update) and resource (test session) plus the two mutable fields (name, pass/fail status), which is enough to separate it from deleteSession and stopSession. It does not explicitly name siblings or explain how it differs from the tm_update_* tools, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The clause 'from your test framework' implies the scenario (reporting an external test result) but there is no explicit when-to-use, when-not-to-use, or named alternative. Usage is inferable rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

upload_appUpload App to LambdaTestBInspect

Uploads an app to LambdaTest for mobile testing and displays upload status.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName for the uploaded app
appUrlNoURL of the app to upload. Provide this OR appFilePath (exactly one is required).
customIdNoCustom ID for the app (optional)
visibilityNoVisibility setting (optional, default: individual)
appFilePathNoAbsolute path to the app file (APK, IPA, etc.). Provide this OR appUrl (exactly one is required).

Output Schema

ParametersJSON Schema
NameRequiredDescription
appIdNoThe app ID
errorNoError message if upload failed
appUrlNoThe URL of the uploaded app to use in capabilities
messageYesStatus message
successYesWhether the upload was successful

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false, and openWorldHint=false, so the agent knows this is a non-destructive write. The description adds that upload status is surfaced, but says nothing about processing latency, async behavior, or whether the uploaded app persists until deleted (see deleteUploadedApps). Adequate but thin against the annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single compact sentence with the verb front-loaded and no filler. It is appropriately sized, though it is too terse to carry any routing or behavioral detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described, and the schema fully documents the five parameters including the enum and the either/or upload source. For a straightforward non-destructive upload tool this is complete, though the sibling ambiguity is a missed opportunity rather than a gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema itself documents the appUrl/appFilePath mutual exclusivity, the enum for visibility, and which fields are optional. The description contributes no additional parameter semantics beyond the schema, so the baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (uploads an app to LambdaTest for mobile testing) and adds the side effect of displaying upload status. However, it does not differentiate itself from the very similar sibling uploadAppVirtualDevice, leaving the agent to guess which upload path applies.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use guidance, no prerequisites, and no mention of alternatives. With siblings like uploadAppVirtualDevice, uploadUserFile, and uploadPrerunFile, the agent gets no signal about which upload tool is appropriate for mobile app testing versus virtual devices or generic files.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

uploadAppVirtualDeviceUpload App for Virtual DeviceCInspect

Upload an app (APK for emulator, IPA for simulator) to LambdaTest for virtual device (emulator/simulator) testing.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesName for the uploaded app
appUrlNoPublic URL of the app to upload (alternative to filePath)
storageNoStorage method (default: url when appUrl provided)
filePathNoAbsolute local path to the app file (APK or IPA)
custom_idNoCustom identifier for the app
visibilityNoAccess level (default: individual)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=false, destructiveHint=false and openWorldHint=false, so the safety profile is covered. The description adds nothing beyond that: it does not mention authentication requirements, that uploads are asynchronous and require a follow-up processing check, size limits, or what happens to an existing app with the same name.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single well-formed sentence that front-loads the verb and resource with no padding. It is appropriately sized, though it is thin enough that the brevity reflects missing detail rather than disciplined editing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries return-value burden and says nothing about the upload response or the resulting app ID. Missing the async-processing handoff to checkAppProcessingStatusVirtualDevice, a near-certain next step in this workflow, is the main gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all six parameters are already documented, including the filePath/appUrl alternative and the two enums. The description only reinforces the accepted file types (APK/IPA), which the schema already encodes. Baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Upload) and resource (an app, APK/IPA) with the destination scope (LambdaTest virtual device testing). It distinguishes itself from the many other upload* siblings by naming the emulator/simulator target, though it never explicitly contrasts with upload_app or uploadUserFile.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No when-to-use or when-not-to-use guidance. The agent gets no signal about preferring this over upload_app (physical devices), nor about how filePath vs appUrl selection relates to the 'storage' enum. Only the schema hints at the url/filePath alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 24 tool updates
    • ChangedautomationQuery13 fields changed
      • changedInput schema / properties / fromdate / description
        Previous value: -"Filter builds from this date (YYYY-MM-DD)"New value: +"Filter builds from this date (YYYY-MM-DD, a real calendar date)"
      • changedInput schema / properties / limit / description
        Previous value: -"Number of results to return (pagination)"New value: +"Number of results to return, 1 to 100"
      • addedInput schema / properties / limit / maximum
        Added value: +100
      • addedInput schema / properties / limit / minimum
        Added value: +1
      • changedInput schema / properties / limit / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / offset / description
        Previous value: -"Offset for pagination"New value: +"Offset for pagination; pass the next offset from a previous response"
      • addedInput schema / properties / offset / minimum
        Added value: +0
      • changedInput schema / properties / offset / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / pageNumber / description
        Previous value: -"Page number for command log pagination"New value: +"Page number for command log pagination, from 1"
      • addedInput schema / properties / pageNumber / minimum
        Added value: +1
      • changedInput schema / properties / pageNumber / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / sessionId / description
        Previous value: -"Session ID — required for app platform log retrieval"New value: +"Session ID: required for session get and for app platform log retrieval. Session get also accepts the test ID instead."
      • changedInput schema / properties / todate / description
        Previous value: -"Filter builds to this date (YYYY-MM-DD)"New value: +"Filter builds to this date (YYYY-MM-DD, not before fromdate)"
    • ChangeddeleteUploadedApps3 fields changed
      • changedInput schema / properties / appIds / description
        Previous value: -"Array of app IDs to delete"New value: +"Array of app IDs to delete (at least one)"
      • addedInput schema / properties / appIds / items / minLength
        Added value: +1
      • addedInput schema / properties / appIds / minItems
        Added value: +1
    • ChangedgetAccessibilityReport1 field changed
      • addedInput schema / properties / include_rules
        Added value: +{
        +  "description": "Return the full rule catalog (about 11 KB) instead of the default ruleCatalog summary of counts per category and disabled rule ids.",
        +  "type": "boolean"
        +}
    • ChangedgetAnalyticsData1 field changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
    • ChangedgetAnalyticsErrors1 field changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
    • ChangedgetFlakyTests1 field changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
    • ChangedgetLiveSessionLogs1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"The active session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
    • ChangedgetSessionScreenshots1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
    • ChangedgetSessionV2CommandLogs1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
    • ChangedgetSessionV2SeleniumLogs1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
    • ChangedgetSessionVideo1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"The session ID"New value: +"The web automation test ID (e.g. DA-WIN-...) or session ID"
    • ChangedgetTestTrends1 field changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."New value: +"Start date, YYYY-MM-DD, within the last 59 days (older results are not served). Defaults to 30 days before toDate."
    • ChangedlistDevicesForTesting4 fields changed
      • addedInput schema / properties / deviceName
        Added value: +{
        +  "description": "Device name to match, case-insensitively as a substring, e.g. \"Galaxy S24\", \"iPhone 15\", \"Pixel\"",
        +  "type": "string"
        +}
      • addedInput schema / properties / limit
        Added value: +{
        +  "description": "Devices per page, 1 to 200 (default 50)",
        +  "maximum": 200,
        +  "minimum": 1,
        +  "type": "integer"
        +}
      • addedInput schema / properties / offset
        Added value: +{
        +  "description": "Devices to skip (default 0)",
        +  "minimum": 0,
        +  "type": "integer"
        +}
      • addedInput schema / properties / platformVersion
        Added value: +{
        +  "description": "OS version to match, e.g. \"14\" (matches 14 and 14.x)",
        +  "type": "string"
        +}
    • ChangedlistHyperExecuteJobs8 fields changed
      • changedInput schema / properties / limit / description
        Previous value: -"Maximum number of jobs to return"New value: +"Maximum number of jobs to return, 1 to 200 (default 10)"
      • addedInput schema / properties / limit / maximum
        Added value: +200
      • addedInput schema / properties / limit / minimum
        Added value: +1
      • changedInput schema / properties / limit / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / offset / description
        Previous value: -"Pagination offset"New value: +"Number of jobs to skip, after the status filter (default 0)"
      • addedInput schema / properties / offset / minimum
        Added value: +0
      • changedInput schema / properties / offset / type
        Previous value: -"number"New value: +"integer"
      • changedInput schema / properties / status / description
        Previous value: -"Filter by job status (e.g., completed, failed, running)"New value: +"Only jobs with this status (e.g. completed, failed, running, skipped), among the 200 most recent"
    • ChangedlistMobileBuilds3 fields changed
      • addedInput schema / properties / limit
        Added value: +{
        +  "description": "Rows per page, 1 to 100 (default 10)",
        +  "maximum": 100,
        +  "minimum": 1,
        +  "type": "integer"
        +}
      • addedInput schema / properties / offset
        Added value: +{
        +  "description": "Rows to skip (default 0)",
        +  "minimum": 0,
        +  "type": "integer"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Only builds with this status, e.g. completed, running, passed, failed",
        +  "type": "string"
        +}
    • ChangedlistMobileSessions4 fields changed
      • addedInput schema / properties / buildId
        Added value: +{
        +  "description": "Only sessions of this build (build_id from listMobileBuilds)",
        +  "type": "string"
        +}
      • addedInput schema / properties / limit
        Added value: +{
        +  "description": "Rows per page, 1 to 100 (default 10)",
        +  "maximum": 100,
        +  "minimum": 1,
        +  "type": "integer"
        +}
      • addedInput schema / properties / offset
        Added value: +{
        +  "description": "Rows to skip (default 0)",
        +  "minimum": 0,
        +  "type": "integer"
        +}
      • addedInput schema / properties / status
        Added value: +{
        +  "description": "Only sessions with this status, e.g. passed, failed, completed, error",
        +  "type": "string"
        +}
    • ChangedlistUploadedApps2 fields changed
      • changedInput schema / properties / level / description
        Previous value: -"Scope: user (default) or team"New value: +"Scope: user (default, apps you uploaded) or team (apps across your organization; sent to the API as level=org)"
      • addedInput schema / properties / level / enum
        Added value: +[
        +  "user",
        +  "team"
        +]
    • Changedsmartui_build_query1 field changed
      • changedInput schema / properties / build_name / description
        Previous value: -"Build name — alternative to build_id for get/gate"New value: +"Build name. get/gate: exact name, an alternative to build_id. list: case-insensitive substring filter, applied client-side to the newest 200 builds (upstream has no name filter)."
    • Changedsmartui_comparison_analyze1 field changed
      • changedInput schema / properties / include_assets / description
        Previous value: -"urls (default) returns asset URLs only. inline additionally attaches the screenshots themselves as image blocks — compared only for lens=pixel, baseline+current+compared for consolidated, none for lens=dom. Use inline when you cannot fetch URLs."New value: +"urls (default) returns asset URLs only. inline additionally attaches the screenshots themselves as image blocks — compared only for lens=pixel, baseline+current+compared for consolidated, none for lens=dom. Images over 1.5 MB (or past 2.25 MB in total) are skipped and listed with the reason; their URLs stay in the result. Use inline when you cannot fetch URLs."
    • Changedsmartui_screenshot_upload1 field changed
      • changedInput schema / properties / screenshot_name / description
        Previous value: -"Name this screenshot (binds it to that baseline) instead of using the file/URL name. Only valid when uploading exactly one image."New value: +"Name this screenshot (binds it to that baseline) instead of using the file/URL name. Sent exactly as given, with no extension added, so pass the baseline name as it appears in SmartUI (SDK and CLI captures have no extension). Only valid when uploading exactly one image."
    • Changedtm_create_test_case1 field changed
      • changedInput schema / properties / test_steps / items / properties / outcome / description
        Previous value: -"Expected result"New value: +"Expected result, shown as the step Outcome in the UI. Use this key: any other step key (such as expected_result) is rejected."
    • Changedtm_get_test_cases1 field changed
      • changedInput schema / properties / priority / description
        Previous value: -"Filter by priority (e.g. Lowest, Low, Medium, High, Highest)"New value: +"Filter by priority (e.g. Lowest, Low, Medium, High, Highest). A value outside the project's priority list is rejected with the valid values."
    • Changedtm_list_configurations1 field changed
      • removedInput schema / properties / project_id
        Removed value: -{
        -  "description": "Optionally scope to a project",
        -  "type": "string"
        -}
    • Changedtm_update_test_case1 field changed
      • changedInput schema / properties / test_steps / items / properties / outcome / description
        Previous value: -"Expected result"New value: +"Expected result, shown as the step Outcome in the UI. Use this key: any other step key (such as expected_result) is rejected."
  2. 23 tool updates
    • RemovedcheckAppProcessingStatusVirtualDevice
    • RemoveddeleteHyperExecuteJob
    • ChangeddeletePrerunFile2 fields changed
      • changedInput schema / properties / filePath / description
        Previous value: -"Stored LambdaTest file path to delete"New value: +"Stored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>)"
      • changedInput schema / properties / file_path / description
        Previous value: -"Stored LambdaTest file path to delete"New value: +"Stored path of the pre-run or post-run file, as uploadPrerunFile returns it (<name>/pre/<file> or <name>/post/<file>)"
    • ChangeddeleteUserFile2 fields changed
      • changedInput schema / properties / filePath / description
        Previous value: -"Stored LambdaTest file path to delete"New value: +"Key of the stored user file, as uploadUserFile returns it (usually the file name)"
      • changedInput schema / properties / file_path / description
        Previous value: -"Stored LambdaTest file path to delete"New value: +"Key of the stored user file, as uploadUserFile returns it (usually the file name)"
    • ChangedgenerateHyperExecuteYAML20 fields changed
      • changedInput schema / properties / appTesting / description
        Previous value: -"Mobile app testing configuration. ONLY for mobile testing (XCUI/Espresso/Appium). DO NOT include this for web automation tests (Selenium, Playwright, Cypress, etc.)."New value: +"Mobile app testing configuration (XCUI, Espresso or Appium). Web automation (Selenium, Playwright, Cypress, etc.) leaves it out."
      • addedInput schema / properties / appTesting / properties / deviceLog / description
        Added value: +"Capture device logs"
      • addedInput schema / properties / appTesting / properties / networkLog / description
        Added value: +"Capture network logs"
      • addedInput schema / properties / appTesting / properties / video / description
        Added value: +"Record a video of each test"
      • addedInput schema / properties / artifacts / properties / mergeArtifacts / description
        Added value: +"Merge the artifacts from all parallel runners into one set"
      • addedInput schema / properties / artifacts / properties / uploadPaths / description
        Added value: +"Paths on the runner to upload as job artifacts, e.g. reports/ or screenshots/"
      • addedInput schema / properties / execution / properties / autosplit / description
        Added value: +"Spread the discovered tests across the parallel runners automatically"
      • addedInput schema / properties / execution / properties / concurrency / description
        Added value: +"Number of tests run in parallel"
      • addedInput schema / properties / execution / properties / maxRetries / description
        Added value: +"Retries per failed test when retryOnFailure is on"
      • addedInput schema / properties / execution / properties / os / description
        Added value: +"Runner operating system"
      • addedInput schema / properties / execution / properties / retryOnFailure / description
        Added value: +"Retry failed tests"
      • addedInput schema / properties / setup / properties / runtime / properties / language / description
        Added value: +"Runtime to install on the runner, e.g. node, python, java"
      • addedInput schema / properties / setup / properties / runtime / properties / version / description
        Added value: +"Runtime version, e.g. 20 or 3.11"
      • changedInput schema / properties / sourcePayload / description
        Previous value: -"Git integration - ONLY include this if the user explicitly requests fetching code from a remote Git repository. Do NOT include this when code is already present locally (which is the default case). Most users run tests on local code and do not need sourcePayload."New value: +"Git integration: fetches the test code from a remote Git repository. Only needed when the code is not already present locally; local code (the usual case) leaves it out."
      • changedInput schema / properties / sourcePayload / properties / accessToken / description
        Previous value: -"Use env var reference: ${GIT_TOKEN}"New value: +"Reference to an environment variable that holds the Git token, e.g. ${GIT_TOKEN}; never the token itself"
      • addedInput schema / properties / sourcePayload / properties / gitProvider / description
        Added value: +"Git host of the repository"
      • addedInput schema / properties / sourcePayload / properties / platform / description
        Added value: +"Source platform; git is the only supported value"
      • addedInput schema / properties / testing / properties / testDiscovery / description
        Added value: +"How HyperExecute finds the tests to run"
      • addedInput schema / properties / testing / properties / testDiscovery / properties / mode / description
        Added value: +"dynamic discovers tests on the runner at run time; static uses the list found before the run"
      • addedInput schema / properties / testing / properties / testDiscovery / properties / type / description
        Added value: +"How tests are discovered: raw runs the command as given, automatic lets HyperExecute detect them"
    • RemovedgetAccessibilityScanIssues
    • RemovedgetAccessibilityScanReport
    • ChangedgetAutomationTestCommandLogs1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"Required for app platform: the session ID of the app test"New value: +"Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)"
    • ChangedgetAutomationTestNetworkLogs1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"Required for app platform: the session ID of the app test"New value: +"Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)"
    • ChangedgetAutomationTestSeleniumLogs1 field changed
      • changedInput schema / properties / sessionId / description
        Previous value: -"Required for app platform: the session ID of the app test"New value: +"Required for app platform: the session_id of the app session, a UUID that listMobileSessions and getAutomationTestDetails return (not the test ID)"
    • RemovedgetPerformanceTestResult
    • RemovedgetScreenshotTestResult
    • RemovedpostSessionAssertions
    • ChangedpostSessionExceptions4 fields changed
      • changedInput schema / properties / data / additionalProperties
        Previous value: -trueNew value: +false
      • changedInput schema / properties / data / description
        Previous value: -"Exception payload to upload"New value: +"The exceptions to attach: {\"exception\": [\"first message\", \"second message\"]}"
      • addedInput schema / properties / data / properties
        Added value: +{
        +  "exception": {
        +    "description": "Exception messages, one string each",
        +    "items": {
        +      "type": "string"
        +    },
        +    "minItems": 1,
        +    "type": "array"
        +  }
        +}
      • addedInput schema / properties / data / required
        Added value: +[
        +  "exception"
        +]
    • ChangedpostTestExceptions4 fields changed
      • changedInput schema / properties / data / additionalProperties
        Previous value: -trueNew value: +false
      • changedInput schema / properties / data / description
        Previous value: -"Exception payload to upload"New value: +"The exceptions to attach: {\"exception\": [\"first message\", \"second message\"]}"
      • addedInput schema / properties / data / properties
        Added value: +{
        +  "exception": {
        +    "description": "Exception messages, one string each",
        +    "items": {
        +      "type": "string"
        +    },
        +    "minItems": 1,
        +    "type": "array"
        +  }
        +}
      • addedInput schema / properties / data / required
        Added value: +[
        +  "exception"
        +]
    • RemovedstopScreenshotTest
    • Changedtm_create_milestone1 field changed
      • changedInput schema / properties / owner / description
        Previous value: -"Milestone Owner — an organization member (email, username, or user id). ASK the user who owns it if unspecified."New value: +"Milestone owner: an organization member (email, username, or user id). Omit for no owner."
    • Changedtm_create_test_run1 field changed
      • changedInput schema / properties / assignee / description
        Previous value: -"Assign every created instance to this organization member — email, username, or user id. If the user has not said who to assign, ASK them (or confirm no assignee) before creating; never guess."New value: +"Assign every created instance to this organization member: email, username, or user id. Omit to leave the instances unassigned."
    • Changedtm_generate_test_cases1 field changed
      • changedInput schema / properties / files / description
        Previous value: -"Optional reference files to ground the generation (requirement docs, specs, data sheets, screenshots). Supported: .pdf, .docx, .txt, .csv, .xlsx, .json, .xml, .jpg, .jpeg, .png, .gif, .bmp, .webp — max 5 files, 10 MB each. Read the file yourself and base64-encode its raw bytes (shell: base64 -i <file>). Prefer small files (up to a few hundred KB) — base64 content passes through your context, so upload larger files via the Test Manager UI instead. Only used when starting a new generation."New value: +"Optional reference files to ground the generation (requirement docs, specs, data sheets, screenshots). Each item carries the file name and its raw bytes base64-encoded in content_base64. Supported: .pdf, .docx, .txt, .csv, .xlsx, .json, .xml, .jpg, .jpeg, .png, .gif, .bmp, .webp; max 5 files, 10 MB each. Large files are better uploaded in the Test Manager UI, since the base64 content travels in the request. Only used when starting a new generation."
    • Changedtm_record_test_results1 field changed
      • changedInput schema / properties / results / items / properties / assignee / description
        Previous value: -"Assign this instance to an organization member (email, username, or user id; \"none\" clears). ASK the user if unspecified."New value: +"Assign this instance to an organization member (email, username, or user id; \"none\" clears). Omit to keep the current assignee."
    • Changedtm_update_test_run1 field changed
      • changedInput schema / properties / assignee / description
        Previous value: -"Assign the run's instances to this organization member — email, username, or user id (\"none\" clears). Scope with apply_to_test_case_ids. ASK the user who to assign if unspecified; never guess."New value: +"Assign the run's instances to this organization member: email, username, or user id (\"none\" clears). Scope with apply_to_test_case_ids. Omit to keep the current assignee."
    • RemoveduploadPrerunFile
    • RemoveduploadUserFile
  3. 16 tool updates
    • ChangedautomationQuery4 fields changed
      • changedInput schema / properties / action / description
        Previous value: -"The action to perform on the resource"New value: +"The read to perform: list, or get one"
      • changedInput schema / properties / action / enum
        Previous value: -[
        -  "list",
        -  "get",
        -  "delete",
        -  "update",
        -  "stop"
        -]New value: +[
        +  "list",
        +  "get"
        +]
      • changedInput schema / properties / buildId / description
        Previous value: -"Build ID — required for build get/delete/update/stop and optional for session list"New value: +"Build ID — required for build get, optional filter for session list"
      • removedInput schema / properties / name
        Removed value: -{
        -  "description": "New name for build update",
        -  "type": "string"
        -}
    • AddeddeleteBuild
    • ChangedgetAnalyticsData4 fields changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date (ISO 8601 format, e.g. 2024-01-01)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."
      • changedInput schema / properties / groupBy / description
        Previous value: -"Dimension to group results by (e.g., browser, os, status)"New value: +"Dimension to group by. Defaults to status."
      • addedInput schema / properties / groupBy / enum
        Added value: +[
        +  "status",
        +  "browser",
        +  "os",
        +  "build",
        +  "device",
        +  "day"
        +]
      • changedInput schema / properties / toDate / description
        Previous value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
    • ChangedgetAnalyticsErrors3 fields changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date (ISO 8601 format)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."
      • changedInput schema / properties / limit / description
        Previous value: -"Maximum number of results to return"New value: +"Maximum failing tests to list, 1 to 200. Defaults to 20."
      • changedInput schema / properties / toDate / description
        Previous value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
    • ChangedgetFlakyTests3 fields changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date (ISO 8601 format)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."
      • changedInput schema / properties / limit / description
        Previous value: -"Maximum number of results to return"New value: +"Maximum flaky tests to return, 1 to 200. Defaults to 50."
      • changedInput schema / properties / toDate / description
        Previous value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
    • ChangedgetHyperExecuteJobInfo2 fields changed
      • removedInput schema / properties / lt_access_key
        Removed value: -{
        -  "description": "LambdaTest access key (optional)",
        -  "type": "string"
        -}
      • removedInput schema / properties / lt_username
        Removed value: -{
        -  "description": "LambdaTest username (optional)",
        -  "type": "string"
        -}
    • ChangedgetHyperExecuteJobSessions2 fields changed
      • removedInput schema / properties / lt_access_key
        Removed value: -{
        -  "description": "LambdaTest access key (optional)",
        -  "type": "string"
        -}
      • removedInput schema / properties / lt_username
        Removed value: -{
        -  "description": "LambdaTest username (optional)",
        -  "type": "string"
        -}
    • ChangedgetSupportedPlatforms3 fields changed
      • addedInput schema / properties / browser
        Added value: +{
        +  "description": "Browser to expand on desktop, e.g. \"chrome\", \"firefox\", \"edge\", \"safari\"",
        +  "type": "string"
        +}
      • addedInput schema / properties / os
        Added value: +{
        +  "description": "OS name to expand, matched case-insensitively as a substring, e.g. \"Windows 11\", \"Sonoma\", \"android\"",
        +  "type": "string"
        +}
      • addedInput schema / properties / type
        Added value: +{
        +  "description": "Only desktop OSes or only mobile platforms",
        +  "enum": [
        +    "desktop",
        +    "mobile"
        +  ],
        +  "type": "string"
        +}
    • ChangedgetTestTrends2 fields changed
      • changedInput schema / properties / fromDate / description
        Previous value: -"Start date (ISO 8601 format)"New value: +"Start date, YYYY-MM-DD. Defaults to 30 days before toDate."
      • changedInput schema / properties / toDate / description
        Previous value: -"End date (ISO 8601 format)"New value: +"End date, YYYY-MM-DD. Defaults to today."
    • Changedsmartui_baseline_manage2 fields changed
      • changedInput schema / properties / action / description
        Previous value: -"get_settings reads baseline config; merge_branch/merge_build merge baselines"New value: +"merge_branch merges one branch baseline into another; merge_build merges one build into another"
      • changedInput schema / properties / action / enum
        Previous value: -[
        -  "get_settings",
        -  "merge_branch",
        -  "merge_build"
        -]New value: +[
        +  "merge_branch",
        +  "merge_build"
        +]
    • Changedsmartui_screenshot_upload1 field changed
      • removedInput schema / properties / project_token
        Removed value: -{
        -  "description": "Explicit projectToken (alternative to project_id)",
        -  "type": "string"
        -}
    • AddedstopBuild
    • Changedtm_list_configurations1 field changed
      • changedInput schema / properties / limit / description
        Previous value: -"Max results per page"New value: +"Max results per page (default 10)"
    • Changedtm_list_projects1 field changed
      • changedInput schema / properties / limit / description
        Previous value: -"Max results per page"New value: +"Max results per page (default 25)"
    • ChangedtriggerSuiteRun2 fields changed
      • removedInput schema / properties / accessKey
        Removed value: -{
        -  "description": "Access key for Basic Auth (optional, uses authenticated credentials if not provided)",
        -  "type": "string"
        -}
      • removedInput schema / properties / username
        Removed value: -{
        -  "description": "Username for Basic Auth (optional, uses authenticated user if not provided)",
        -  "type": "string"
        -}
    • AddedupdateBuild
  4. 95 tool updates
    • First observedanalyzeAppViaTunnel
    • First observedanalyzeHyperExecuteJob
    • First observedanswerHyperExecuteQuery
    • First observedautomationQuery
    • First observedbuildLocalAppForAnalysis
    • First observedcheckAppProcessingStatus
    • First observedcheckAppProcessingStatusVirtualDevice
    • First observeddeleteHyperExecuteJob
    • First observeddeletePrerunFile
    • First observeddeleteSession
    • First observeddeleteUploadedApps
    • First observeddeleteUserFile
    • First observeddownloadJobArtifact
    • First observedgenerateHyperExecuteYAML
    • First observedgetAccessibilityReport
    • First observedgetAccessibilityScanIssues
    • First observedgetAccessibilityScanReport
    • First observedgetAnalyticsData
    • First observedgetAnalyticsErrors
    • First observedgetAutomationTestBrowserConsoleLogs
    • First observedgetAutomationTestCommandLogs
    • First observedgetAutomationTestDetails
    • First observedgetAutomationTestNetworkLogs
    • First observedgetAutomationTestSeleniumLogs
    • First observedgetFlakyTests
    • First observedgetHyperExecuteJobArtifacts
    • First observedgetHyperExecuteJobInfo
    • First observedgetHyperExecuteJobLiveStream
    • First observedgetHyperExecuteJobSessions
    • First observedgetHyperExecuteStage
    • First observedgetHyperExecuteStageLogs
    • First observedgetHyperExecuteStageTasks
    • First observedgetHyperExecuteTask
    • First observedgetHyperExecuteTaskLogs
    • First observedgetJobArtifactDetail
    • First observedgetLiveSessionLogs
    • First observedgetMobileCapabilityGenerator
    • First observedgetMobileConcurrency
    • First observedgetOrgConcurrency
    • First observedgetPerformanceTestResult
    • First observedgetScreenshotTestResult
    • First observedgetSessionScreenshots
    • First observedgetSessionV2CommandLogs
    • First observedgetSessionV2SeleniumLogs
    • First observedgetSessionVideo
    • First observedgetSuiteOverview
    • First observedgetSupportedPlatforms
    • First observedgetTestTrends
    • First observedlistDevicesForTesting
    • First observedlistHyperExecuteJobs
    • First observedlistMobileBuilds
    • First observedlistMobileSessions
    • First observedlistTunnels
    • First observedlistUploadedApps
    • First observedpostSessionAssertions
    • First observedpostSessionExceptions
    • First observedpostTestExceptions
    • First observedsmartui_baseline_manage
    • First observedsmartui_build_query
    • First observedsmartui_comparison_analyze
    • First observedsmartui_comparison_query
    • First observedsmartui_project_query
    • First observedsmartui_screenshot_review
    • First observedsmartui_screenshot_upload
    • First observedstopScreenshotTest
    • First observedstopSession
    • First observedstopTunnel
    • First observedtm_create_folder
    • First observedtm_create_milestone
    • First observedtm_create_project
    • First observedtm_create_test_case
    • First observedtm_create_test_run
    • First observedtm_generate_test_cases
    • First observedtm_get_coverage_summary
    • First observedtm_get_milestone
    • First observedtm_get_test_cases
    • First observedtm_get_test_runs
    • First observedtm_link_ado_issue
    • First observedtm_link_jira_issue
    • First observedtm_list_configurations
    • First observedtm_list_folders
    • First observedtm_list_modules
    • First observedtm_list_projects
    • First observedtm_record_test_results
    • First observedtm_update_folder
    • First observedtm_update_milestone
    • First observedtm_update_project
    • First observedtm_update_test_case
    • First observedtm_update_test_run
    • First observedtriggerSuiteRun
    • First observedupdateSession
    • First observedupload_app
    • First observeduploadAppVirtualDevice
    • First observeduploadPrerunFile
    • First observeduploadUserFile

Related MCP Connectors

Related MCP Servers

  • F
    license
    Not graded
    quality
    C
    maintenance
    Exposes AI agents as MCP tools to fetch Jira stories, generate BRDs and Playwright test scripts, run tests, and auto-heal broken locators. Integrates with Claude Desktop and Cursor IDE for enterprise-grade test automation.
    -
  • F
    license
    Not graded
    quality
    C
    maintenance
    MCP server for the Argus E2E Testing Agent, enabling AI coding assistants to perform browser testing, quality intelligence, CI/CD integration, discovery, time travel debugging, visual AI testing, correlation analytics, and API testing directly from the IDE.
    -
  • A
    license
    Not graded
    quality
    D
    maintenance
    A universal AI-powered testing server built on the Model Context Protocol (MCP). Allows AI agents to inspect, execute, test, monitor, debug, and report on software projects.
    3
    GNU Lesser General Public v2.1 only
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources