TestMu AI Test Manager MCP

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.1.0
Disambiguation4/5
Tools are mostly distinct with clear resource+action targets. Some potential confusion between 'get_testCaseInstancesByTestRunId' and 'get_testCaseInstanceById' (collection vs single), but descriptions explicitly distinguish IDs and granularity. Overlap is minimal.
Naming Consistency4/5
Predominantly follows 'tm.<verb>_<noun>' pattern (e.g., create_testCases, get_testRunById). Inconsistency with 'bulkUpdate_testCaseInstances' using camelCase for 'bulkUpdate' while others use snake_case (e.g., 'add_testCasesToTestRun'). Overall convention is mostly consistent.
Tool Count3/5
39 tools is high for a single server, covering Test Manager, HyperExecute, and AI RCA. Many are read-only, but the count feels heavy. Tools are scoped to a specific domain, but some granularity could be merged (e.g., multiple get_hyperExecute* tools).
Completeness3/5
Covers main CRUD for test artifacts and execution lifecycle, plus Jira linking and RCA. Notable gaps: no deletion tools for projects/folders/test cases/runs, no listing of all projects (only by ID), and no direct update of existing test steps (only append). Core workflows are present but missing cleanup operations.
Average 4.6/5 across 39 of 39 tools scored. Lowest: 3.8/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 4 commits in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI status not available
Add a LICENSE file by following GitHub's guide. Once GitHub recognizes the license, the system will automatically detect it within a few hours.
If the license does not appear after some time, you can manually trigger a new scan using the MCP server admin interface.
MCP servers without a LICENSE cannot be installed.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that creating is a real, persistent action, but does not mention authentication requirements, rate limits, or what is returned. For a mutation tool, more detail on side effects would be beneficial.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with 3 sentences that front-load the core action, then add usage context and warnings. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple creation tool with no output schema and no annotations, the description covers the main purpose and usage context. However, it lacks mention of return values, error conditions, or idempotency, which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must provide meaning. It explains project_id and parent_id, but the required name parameter and the optional description parameter are not described. This provides partial but incomplete coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states it creates a new folder in a LambdaTest Test Manager project, specifies required project_id and optional nesting via parent_id, and distinguishes from updating or moving folders. It also contrasts with sibling tool tm.create_testRunFolder by focusing on test case folders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clear guidance to use before adding test cases, with explicit warnings against speculative calls and against using it for update/move. However, it doesn't directly compare with the very similar sibling tm.create_testRunFolder, which could cause confusion.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Describes returned data and usage but does not explicitly state it is a read-only, non-destructive operation or mention auth requirements or rate limits. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph, front-loaded with main purpose, then adds use cases and behavior. Some redundancy but overall informative and well-structured. Could be slightly more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description thoroughly explains return fields (name, ID, description, parent, timestamps, test case counts) and their significance. Relates to sibling tools and provides actionable context. Complete for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only parameter is project_id, described in schema as string with minLength. Description mentions 'given the project's ID' but adds no further detail about expected format, source, or validation. With 0% schema coverage, more explicit guidance would be beneficial.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves every folder in a LambdaTest Test Manager project by project ID, listing returned fields (name, ID, description, parent folder, timestamps, test case counts). It distinguishes from sibling tm.get_testCasesByFolderId by mentioning total counts including subfolders.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit use cases: to see test case organization, find folder ID before adding test cases, spot count discrepancies. Also says 'Do not use this to fetch a single folder's details,' giving a clear exclusion. Lacks mention of alternative for single folder details but implied via siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explains the tool returns orientation and rules, but does not explicitly state it is read-only or has no side effects, though that is implied by the content.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, front-loaded with main purpose. Efficient but could better structure the parameter information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description should fully specify return content. It does so for the main content but omits any mention of the input parameter, reducing completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has one optional parameter 'name' with no description coverage. The tool description does not mention or explain this parameter, leaving its purpose unclear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it returns a short orientation for the server, including what it wraps and rules. It distinguishes itself from sibling tools that perform specific operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to call first if unfamiliar with the server, and notes it does not enumerate tools, directing to the tool catalog. Provides clear when-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full burden. It explicitly states the tool is read-only and does not modify anything. It also clarifies the aggregation behavior of planned_executions_count and explains the automation_test_id field for RCA, which adds useful behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the core purpose in the first sentence, then expands with useful details. It is somewhat lengthy but well-structured into paragraphs. Each part earns its place, though minor trimming could improve conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains return fields thoroughly (status, run, timestamps, environment, counts). It also clarifies edge cases like planned_executions_count aggregation. It lacks mention of pagination or error handling, but covers the main use case well.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has one parameter (test_case_id) with no description (0% coverage). The description only says 'by its exact test case ID' without specifying format or source, adding minimal value over the schema. For a single parameter, more guidance was expected.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the tool retrieves execution history for a given test case ID, listing detailed information like status, run, environment, etc. It distinguishes itself from siblings like get_testExecutionHistoryByJiraId by focusing on test case ID and providing specific return fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states 'use this to inspect how a test case has performed over time,' giving clear context. It does not explicitly mention when not to use it or compare to alternatives, but provides detailed guidance on the returned data and interpretation of planned_executions_count.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but description fully covers the read-only nature and specifies returned fields. No hidden behaviors disclosed; adequate for a simple retrieval.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, front-loaded purpose, each sentence adds value. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 1-param retrieval, description covers purpose, usage, and output fields. Missing error handling notes, but acceptable given simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, description only says 'exact project ID' without format details or constraints. Minimal added value over schema's param name.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the tool retrieves a single project by exact ID, lists returned fields (name, description, test case count, etc.), and explicitly differentiates from search/list tools, which aligns with the sibling context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use scenarios (verify existence, show metadata) and when-not-to (search by name, list all). Lacks explicit sibling names but still clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses the tool is undocumented, best-effort, changes only run status (not per-instance), and is a persistent action. More detail on failure modes would improve, but current disclosure is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences, each serving a distinct purpose: action, caveat (undocumented), and warning (persistent). No wasted words. Front-loaded with main function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks description of return values or error handling. Given no output schema, the description should ideally mention what the tool returns on success/failure. Current description provides enough to use the tool but incomplete for full understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must add meaning. It lists allowed status values and clarifies that test_run_id identifies the run. However, it does not explain the format or source of test_run_id, leaving some ambiguity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets the overall status of a test run to specific values, distinguishing it from per-instance status updates. Sibling tools like tm.update_testCaseInstance handle instance-level, so this is well differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: 'Do not call this speculatively - it's a real, persistent action.' It also warns that the endpoint is undocumented. However, it does not explicitly contrast when to use this versus alternative tools, though the scope is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that creating a project is a 'real, persistent action,' implying side effects, but lacks details on authorization, rate limits, or error conditions. Adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded, no filler. Every sentence adds value: first states purpose and parameters, second gives usage guidance. Highly efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the number of sibling tools and no output schema, the description adequately covers creation behavior. It explains when to use and the persistent nature. Could mention unique name constraints or response format, but acceptable for a create tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It mentions the three parameters (name, optional description, tags) and clarifies that tags can be zero or more. However, it does not add format constraints or examples, providing only basic mapping to schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates a new LambdaTest Test Manager project, specifying the resource (project) and the fields (name, description, tags). It distinguishes from sibling tools by indicating this is for starting new projects, not updating or other operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (user wants a new project) and when not to (do not update existing project, do not call speculatively). Provides clear guidance against misuse, which is rare and valuable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses that the action is real and persistent, and that the folder tree is separate from test case folders. However, it does not cover authentication requirements, rate limits, or error handling, which would be beneficial for a complete behavioral picture.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that efficiently conveys the main action and additional instructions. It is front-loaded with the core purpose and provides necessary context without extraneous details. Minor improvement could be to structure as bullet points for easier scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description covers the tool's functionality, usage context, and constraints well. It explains the sibling differentiation and cautions against misuse. It does not describe the return value or error scenarios, but for a creation tool, the provided context is largely sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% parameter description coverage, so the description must compensate. It explains project_id and name as required, parent_id as optional (linking to a sibling tool), but does not mention the description parameter. While it adds value, the omission of one parameter leaves a gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the tool creates a new folder for organizing test runs within a LambdaTest Test Manager project. It distinguishes itself from tm.create_folder by clarifying that it manages a separate folder tree for test runs, not test cases, ensuring the agent can differentiate between similar tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use the tool: to create a folder for test runs, with optional nesting via parent_id from tm.get_testRunFoldersByProjectId. It also clearly states what not to do (update, move, or speculatively call) and emphasizes that the action is persistent, aiding decision-making.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses read-only nature, pagination, filtering, and include_run_count behavior. However, it does not mention rate limits, data freshness, or response size limits, which are relevant for a list tool. Additional behavioral details would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loading the purpose. The first sentence clearly states the action and scope. The second adds cross-reference, and the third provides usage guidance. No wasted words; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description explains the output's key field (environment_id) and its connectivity. It covers all parameters and provides usage guidance. However, it could mention that no parameters are required and possibly describe the output structure in more detail.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, so the description must compensate. It explains each filter parameter (browser, os, platform, resolution), pagination (page, per_page), and include_run_count, and crucially states that environment_id in output is usable in other tools. This adds substantial meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists environment configurations (browser/OS/device/resolution) at the org level, distinguishing it from sibling tools that operate on test runs, test cases, or projects. The verb 'lists' and resource 'environments' are specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context: it's read-only, supports pagination and filtering, and advises using filters to avoid paging through large lists. It also explains how the output environment_id connects to other tools. However, it does not explicitly state when not to use it or list alternatives, which would strengthen the score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the burden. It explicitly states the tool is read-only, explains the entry semantics (test case x environment pairing), details automation-specific fields, and describes pagination/filtering behavior. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured: purpose first, then terminology clarification, sibling comparison, automation details, filtering, and read-only note. Every sentence adds value, though it could be slightly tighter. Front-loaded with core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and complex parameter set, the description covers the output structure (breakdown + per-instance entries with fields), automation extras, pagination, and filtering. Lacks explicit mention of default pagination values or status format strictness, but overall sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must add meaning. It explains test_run_id implicitly, notes pagination (page, per_page) and filtering by status and assignee, and gives example status values. However, it does not specify default values or constraints like max per_page, and assignee is only mentioned as user ID without further detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Retrieves') and resource ('test case instances by test run ID'). It distinguishes itself from the sibling tool tm.get_testRunById by contrasting planned composition vs actual results, and from other list tools by its specific scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit context on when to use: for actual execution results vs tm.get_testRunById for planned composition. Mentions filtering and pagination options, guiding the agent on parameter usage. Does not list all alternatives but the single sibling comparison is effective.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that linking is a persistent action, explains the observable effect on the test case's links and query results, and notes the org requirement. However, it does not detail error conditions or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is only four sentences, each adding distinct value: purpose and effect, required params, org prerequisite, and caution against speculative use. No wasted words and the key action is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given low complexity and no output schema, the description adequately covers purpose, prerequisites, effect, and a caution. It lacks mention of return value or error handling, but the context is mostly complete for selection and safe invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It provides a format example for jira_issue_id ('PROJ-123') and states project_id and test_case_id are required, but does not explain their purpose, how to obtain them, or expected formats beyond minLength.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool links a Jira issue to a test case, specifies the effect on the test case's Jira links and the get_testExecutionHistoryByJiraId, and implicitly distinguishes from the sibling unlink_jiraIssue tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Specifies required parameters (project ID, test case ID), prerequisite that the Jira issue must belong to the LT_ORG_ID org, and explicitly warns against speculative calls by stating the action is real and persistent. This provides both when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Describes what data is returned and pagination support; no annotations provided. Could mention read-only nature or side effects, but is generally transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with essential information, no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, description covers purpose, returned fields, usage guidance, and optional parameters; lacks detail on return format but is sufficient for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%; description adds context that page, per_page, sort are optional pagination/sorting parameters, but lacks details like sort format or valid values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it retrieves every test case across a project and lists returned fields. It distinguishes from sibling tools that get test cases by folder.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises use for project-wide view and suggests alternative tools for folder-specific needs, plus mentions optional pagination parameters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It declares the tool as read-only ('does not modify anything') and lists what it returns. It could mention potential error cases or response size limits, but the information given is sufficient for basic transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that front-loads the main purpose, then adds distinctions and usage notes. Every sentence contributes useful information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description explains the return fields (name, ID, timestamps, counts) and clarifies the separation from test case folders. It does not specify if results are paginated or ordered, but for a tool with one parameter, essentials are covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds context that the project_id refers to a LambdaTest project, but it does not elaborate on format or source (e.g., from getProjects). This provides marginal added value over the schema's parameter definition.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Retrieves' and clearly identifies the resource as the folder/subfolder hierarchy for test runs. It distinguishes from the sibling tool tm.get_foldersByProjectId for test case folders, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: to get test run folder hierarchy. It also clarifies what it does not return (the runs themselves) and directs to tm.get_testRunsByProjectId for listing runs, providing an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully handles behavioral disclosure. States read-only, does not modify, explains effect scope (run-level only, not underlying test case status), and lists included fields (result, timing, remarks, environment, per-step results).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is somewhat long but each sentence adds value. Front-loaded with purpose, then terminology clarification, then detail on output and behavior. Could be slightly more concise, but efficient for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description provides adequate overview of returned data: instance result, timing, remarks, environment, per-step results with own status/outcome/remarks and step IDs. Lacks explicit mention of all possible fields, but sufficient for understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must explain parameter. It defines test_instance_id as numeric ID, distinguishes from test_case_id, and tells how to get it (from sibling tool's 'Instance ID' field). Adds value beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it retrieves full detail for one specific test case instance, clarifies terminology differentiating from test case, and contrasts with sibling tool tm.get_testCaseInstancesByTestRunId which lacks this detailed output.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use (need full detail for one instance), contrasts with sibling tool, and states read-only nature. Could be improved by explicitly listing when not to use, but clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full burden. It discloses that only direct test cases are returned, lists the fields in each result, and mentions pagination support. However, it does not specify error behavior or empty results, which is a minor gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph that front-loads the key purpose and exclusions. It is slightly lengthy but efficient. There is minor redundancy in mentioning the prerequisite twice, but overall it earns its sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description partially explains return fields. It includes critical context about subfolder exclusion and prerequisite. However, it does not cover error handling or pagination response format, which would be beneficial for completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description provides the only explanation. It clarifies that project_id and folder_id are required, and page, per_page, and sort are optional. It does not detail the sort parameter format, but the context of pagination is well covered.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states the verb (retrieves), resource (test cases in a folder), and scope (directly assigned, not subfolders). It also lists the fields returned, which distinguishes it from siblings like get_testCaseById or get_testCasesByProjectId.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear when-to-use guidance (retrieving test cases in a folder) and when-not-to-use (not for a single test case, not for subfolder contents). It also gives a prerequisite: use get_foldersByProjectId first to get the folder ID. Pagination options are mentioned for large folders.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavioral traits: only provided fields are changed, at least one field is required, the persistence of the action, and a critical warning about environment_id corruption causing 500 errors. This is comprehensive and transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured, starting with the core action, then listing fields, then explaining identifiers and sources, and ending with a strong warning. Every sentence adds value, but it is slightly lengthy; could be tightened without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description provides enough context for using the tool: how to get parameters, what the tool does, and a crucial safety warning. It lacks error handling details (e.g., if test_instance_id doesn't exist) but is otherwise complete for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema description coverage is 0%, the description adds significant meaning by explaining each parameter's purpose, valid values (status enum), and how to obtain valid IDs (assignee from tm.get_organizationUsers, environment_id from tm.get_environments or existing instances). It compensates well for the lack of schema descriptions, though not every parameter is covered in equal depth.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'updates' and resource 'test case instance's own fields', listing specific fields (status, assignee, environment_id, remarks) and the identifier (test_instance_id). It distinguishes from siblings like tm.bulkUpdate_testCaseInstances by focusing on a single instance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to update specific fields of a single instance), provides sources for parameters (e.g., tm.get_organizationUsers for assignee, tm.get_environments for environment_id), and warns against speculative calls. However, it does not explicitly contrast with sibling tools like tm.bulkUpdate_testCaseInstances or tm.update_testCaseInstanceStep.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that creating test cases is a real, persistent action (destructive behavior), mentions batch operation, and explains optional folder_id default behavior. However, it does not mention authorization requirements, rate limits, or return format, which are minor gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph but well-structured: starts with main purpose, details parameters, then provides usage guidelines. Slightly verbose in places but effective. Could be broken into separate sentences for clarity, but overall it is concise enough.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description covers core behavior, parameter meanings, usage guidance, and side effects. However, it does not explain what the tool returns (e.g., IDs of created test cases), which would be helpful. This is a minor gap, but the description is otherwise rich.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (0% coverage). The description adds significant meaning: explains project_id is required, folder_id optional with guidance to find it, and details the test_cases array structure with required title and optional description, preconditions, tags. This fully compensates for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates one or more test cases in a LambdaTest Test Manager project. It specifies the action (creates), resource (test cases), and context (single batch call). It distinguishes from siblings like tm.update_testCase by specifying 'do not use this to update an existing test case'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance: use when user wants to add new test cases, do not use for updates, and do not call speculatively because it is a real, persistent action. It also suggests using tm.get_foldersByProjectId to find folder ID, offering a clear alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It details return content and the importance of snapshot_id for updates, but does not mention potential errors or authorization needs. Overall, good behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the main action and well-structured, but the exhaustive list of return fields makes it slightly verbose. It efficiently communicates key information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool (1 param, no output schema), the description is highly complete. It covers return content, usage context, prerequisite for update, and differentiation from similar tools. No critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It implicitly defines the single parameter by stating 'by its exact test case ID', but could be more explicit about format or validation. Still, sufficient for understanding parameter purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves full details of a test case by ID, lists specific fields, and explicitly distinguishes from search/list tools, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says when to use (known ID, inspect content, prerequisite for update) and when not (search/list). Provides clear usage context and exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses danger of invalid folder_id causing 500 error, that the run is created with zero test cases, that it's persistent, and that KaneAI runs have matching constraints. It also says 'safe to retry' on error. This is thorough and transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but each sentence adds value. It is front-loaded with the primary purpose, then details optional args, then behavioral notes, then KaneAI specifics. Slightly verbose but well-structured. Could be tightened slightly without losing content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (6 params, no output schema, no annotations), the description covers the tool's purpose, usage, dangers, and relationships to siblings. It explains the workflow (create shell, then add test cases). Missing explicit mention of return value (no output schema), but otherwise complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so description must compensate. It explains title (required), objective (optional), tags (zero or more), folder_id (optional, from specific tool), and is_auteur_generated (boolean for KaneAI). However, project_id is not explicitly described beyond being required, and 'tags' format could be more detailed. Still, it adds significant value over the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates just the shell of a test run, specifying the resource and action. It distinguishes itself from sibling tools like tm.add_testCasesToTestRun by noting that test cases are added separately. The verb 'creates' is specific and the resource 'test run shell' is well-defined.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when to use this tool (to create the shell) and when not to (add test cases later via PUT). It warns against speculative use due to persistence, and advises using tm.get_testRunById afterward to confirm. It also names an alternative for folder_id (tm.get_testRunFoldersByProjectId) and distinguishes manual vs KaneAI runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses read-only nature, snapshot semantics, and defensive parsing of jobLabel. It details output structure and notes no modifications. Minor gaps like error handling or rate limits prevent a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is informative and well-structured with clear sections, but slightly lengthy. It front-loads the main purpose and adds important details without redundancy. Could be trimmed slightly without losing meaning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description comprehensively covers the return structure (job-level info, per-Task breakdown, summaries) and operational behavior (snapshot, defensive parsing). Complete for a read-only retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description thoroughly explains the job_id parameter: UUID format, distinction from other IDs, and requirement to already be known. Includes an example, adding significant value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves current status and full detail of a HyperExecute Job by job ID, listing specific data categories. It distinguishes itself from siblings like get_hyperExecuteJobs (list) and get_hyperExecuteJobScenarios (scenarios) by focusing on a single job with full details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly specifies that input is a known job_id (UUID format, distinct from other IDs), stating this tool does not discover IDs from Test Manager. It also advises calling again for updated status on queued/running jobs, providing clear when-to-use and snapshoting behavior.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Clearly explains read-only behavior, paging mechanism, null possibility of show_test_summary, and linking to Test Manager. Very transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with listing, parameters, and important notes, but slightly verbose at over 150 words. Could be trimmed slightly without losing clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but description lists returned fields. Explains paging challenges and test_run_id linking. For a read-only listing tool with 3 params, this is complete and comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but description explains each parameter in detail: limit (page size default 10), cursor (resume from job_number), show_test_summary (request field with null behavior). Adds substantial meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Describes listing all HyperExecute jobs in organization, specifying returned fields, and distinguishes from sibling tm.get_hyperExecuteJobById which targets a single job.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
States the tool is for listing many jobs not scoped to one, and provides guidance on paging due to lack of filtering by test_run_id. Could be more explicit about when not to use, but context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully covers behavioral traits: it declares read-only nature, explains how status/search_text filters return errors instead of empty lists, notes that tests without sessions are excluded, and describes cursor-based pagination. This provides complete transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with front-loaded purpose and differentiation. It is dense with valuable information, and every sentence adds utility. Minor opportunity to tighten, but overall effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description thoroughly describes each entry's fields (session/test ID, parent Task ID, scenario name, status, group number, duration, SmartUI). It also covers pagination, error handling, and parameter behavior. The tool is moderately complex and fully covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description explains all five parameters in detail: job_id (required, same ID as used by related tools), limit (max 20, default 10), cursor (pagination behavior), status (filter), and search_text (filter by scenario name). It compensates fully for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists session-level execution details for a HyperExecute Job, and explicitly distinguishes from the sibling tool tm.get_hyperExecuteJobScenarios by explaining the difference in how retried tests and failed-to-session tests are handled. This is specific and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use this tool versus alternatives, explicitly comparing with tm.get_hyperExecuteJobScenarios and noting differences. It also explains the input job_id and optional parameters. It lacks an explicit 'when not to use' but the guidance is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully covers behavioral traits: it declares the tool is read-only, explains the prerequisite that the instance must have reached a session, and clarifies that failures before session creation have no automation_test_id. This transparency is thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise yet comprehensive, with a clear first sentence, a list of returned fields, and usage guidance. It could be slightly more compact, but it is well-structured and every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema, the description adequately lists what is returned. It also contextualizes the tool within the sibling tools and explains the relationship. The complexity is well-addressed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single parameter (automation_test_id) has 0% schema description coverage, but the description adds substantial meaning: it identifies the ID as the same shown by other tools and explains how to obtain it. This fully compensates for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: retrieving HyperExecute routing details for a specific automation test execution by its automation_test_id. It lists the returned fields (status, Job ID, etc.) and distinguishes itself from siblings like tm.get_hyperExecuteJobSessions and tm.get_hyperExecuteJobById by focusing on a single execution's details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends this tool as the way to find the HyperExecute Job for an automation_test_id, provides prerequisites (ID must be known from other tools), and notes limitations (instance must have reached a session). It lacks explicit when-not-to-use guidance, but the context is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description fully covers behavior. States it is read-only and does not modify anything. Also discloses that it calls an undocumented endpoint on a different service (auth.lambdatest.com) sourced from browser network inspector, adding important context about reliability and source.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three sentences: first states purpose and output, second gives usage guidance, third provides behavioral transparency. No unnecessary words, efficiently front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description explains return fields thoroughly. Also covers read-only nature and undocumented endpoint. Complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters, schema coverage 100%, so baseline is 4. Description adds value by listing the fields returned (user ID, name, email, role, group, TMS access), which is helpful since there is no output schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists every user in the LambdaTest organization with specific fields (numeric user ID, name, email, role, group, TMS access). Distinguishes itself from siblings by specifying its role in user ID lookup for assignment in other tools like tm.add_testCasesToTestRun.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly says to use this tool to look up a user's ID before assigning them via the assignee field on other tools. Provides clear context but does not explicitly mention when not to use or alternative tools, though no alternative exists among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. It explicitly states 'Read-only; does not modify anything', and discloses behavior for missing data ('returns an empty history rather than an error'). This provides clear expectations without annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first details what is retrieved and the output fields, second states usage and behavior (empty history, read-only). No fluff, all information earns its place, front-loaded with key action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given one parameter, no output schema, and no annotations, the description provides rich detail: output fields, error handling, read-only nature, and usage context. It is fully adequate for correct tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter `jira_issue_id` with 0% schema description coverage. The description adds value by explaining the format ('e.g. 'PROJ-123'') and context, which is not present in the schema (only type and required). This compensates for the lack of schema description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Retrieves' and resource 'execution history of every LambdaTest Test Manager test case linked to a given Jira issue ID', provides an example format ('PROJ-123'), and lists output fields. It distinguishes from sibling `get_testExecutionHistoryByTestCaseId` by specifying Jira ID linkage.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells when to use ('Use this to see how all test cases tied to a Jira ticket have performed') and describes behavior when no linked executions (returns empty history, not error). While it does not explicitly state when NOT to use or suggest alternatives, the purpose is clear enough to differentiate from sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavior: read-only nature, API quirk detailing how the tool corrects field inconsistencies, and clarifies that shown status is review status not execution outcome.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but efficient: every sentence adds unique value (inclusions, exclusions, terminology, quirk, cross-tool notes). Well-structured with front-loaded main purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and a simple input schema, the description thoroughly covers return value details, behavioral quirks, and cross-referencing with sibling tools, making the agent fully informed for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The sole parameter 'test_run_id' is simple, but the description adds no extra semantics beyond the schema's type and minLength. While the parameter is self-explanatory, given 0% schema coverage, the description could have elaborated on format or examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves a test run by ID, lists exactly what data is included (title, objective, status, type, build state, tags, test cases with environments) and explicitly notes what is excluded (execution results). It differentiates from siblings like tm.get_testRunsByProjectId and tm.get_testExecutionHistoryByTestCaseId.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use (needing single run details) and when-not (for execution results, directing to alternative tool). Also provides terminology clarifications and cross-tool consistency checks, giving comprehensive usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description fully discloses destructive nature (spends real credits, cannot be undone), automatic skipping of existing/generating RCAs, mixed batch routing, all-or-nothing dispatch on insufficient credits, and rejection of scopes >10,000 failed tests. Highly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is long but every sentence adds value. Front-loaded with main action, then details constraints and dangers. Could be slightly more concise, but structure is logical and complete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description explains return values (new triggers vs skipped, credits used). Covers error cases (402, 413) and safety checks. Complete for a complex, high-risk tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage, description explains each parameter (job_ids, stage_ids, etc.) as arrays capped at 100 IDs, with at least one required. Adds context on how test_ids route automatically. Could be more explicit about ID format, but compensates well for missing schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it dispatches AI-powered RCA generation for failed tests under a given scope. The title 'Trigger AI Root Cause Analysis Generation' is specific. It distinguishes from sibling tools like tm.get_testExecutionRCA by focusing on generation rather than retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use: for failed tests needing RCA. Also states when-not-to-use: not speculatively or on broad scope. Advises using tm.get_testExecutionRCA first to check existence. Warns about credit costs and all-or-nothing behavior, guiding safe usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, but the description fully discloses read-only nature and explains error behaviors: distinguishes between 'no scenarios match filter' and 'job not found' errors. This provides critical behavioral transparency beyond basic read-only hint.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence adds value: first sentence summarizes output, second details parameters, third explains important error distinction. No redundant or vague language. Front-loaded with main purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 5 parameters and no output schema, the description covers input semantics, pagination, filter behavior, error differentiation, and output fields. It is complete enough for an AI agent to use the tool correctly without additional documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds comprehensive meaning: explains job_id as required and shared with another tool, limit with max 20 and default 10, cursor for pagination (ID-based), status and search_text filters with error handling for no matches.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists scenario-level execution details for a HyperExecute Job, specifying exact fields like scenario ID, parent Task ID, name, iteration, status, group, and duration. It distinguishes from sibling tools like tm.get_hyperExecuteJobById and tm.get_hyperExecuteJobSessions by focusing on scenarios.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description indicates when to use this tool (to get scenario details for a job) and specifies required input (job_id) and optional filters/pagination. It could explicitly mention when not to use or compare to alternatives, but the context is clear enough for an AI agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description fully discloses behavioral traits: read-only nature, pagination mechanics (cursor/limit, max 500), sorting options, date range defaults and constraints (last 7 days default, must provide both timestamps, max 31 days), and filter ID count cap (100 total). No annotation contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured, front-loading the core purpose and then detailing parameters and constraints. Every sentence adds value, though it could be slightly more concise by grouping related constraints. Still, it avoids fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (11 params, no output schema), the description is very comprehensive, covering pagination, sorting, filtering constraints, date behavior, and distinguishing from siblings. It lacks explicit mention of the response format, but the listed fields give a good picture. Overall highly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description adds meaning to all 11 parameters. It explains limit maximum, cursor pagination, filter arrays, sort_by/sort_order enums, and crucially details date parameter constraints (RFC3339 UTC, required together, max span). It also adds undocumented constraints like the total ID count cap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves paginated test execution records enriched with AI insights, listing many specific fields. It distinguishes from sibling tools like tm.get_testExecutionRCA, which provides full RCA detail, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance on when to use this tool vs alternatives (e.g., use tm.get_testExecutionRCA for full RCA detail). Also details filters, date range defaults and constraints, pagination, and sorting, helping the agent decide appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavioral traits: it is read-only, explains the KaneAI-generated run behavior, folder ID inconsistencies, and terminology nuances. This provides complete transparency beyond what structured fields could offer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with sections (KANEAI RUNS, FOLDERS, TERMINOLOGY) and front-loaded with the core purpose. While every sentence adds value, some details could be slightly more concise. Overall, it's appropriate for the complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (6 parameters, no annotations, no output schema), the description is exceptionally complete. It covers edge cases, caveats, and relationships with sibling tools, leaving no significant gaps for an agent to understand correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It adds meaning to parameters like folder_id (explaining it's the test run's own folder) and status (giving examples like 'Not Started', 'Passed'). It also mentions pagination parameters. However, it does not list all parameters explicitly, so not a perfect 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Retrieves every test run in a LambdaTest Test Manager project' and lists the fields returned, making the purpose clear. It also distinguishes from siblings by mentioning alternatives for detail, like tm.get_testRunById and tm.get_testCaseInstancesByTestRunId.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool versus alternatives, such as 'Use tm.get_testRunById or tm.get_testCaseInstancesByTestRunId for full detail on a specific run.' It also includes specific scenarios (e.g., checking 'KaneAI-Generated' before calling tm.add_testCasesToTestRun) and describes filtering options.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It does so by stating 'performs a real upload', that the URL is time-limited, and that the file_key is the persistent identifier. However, it does not mention file size limits, allowed file types, or authentication requirements, leaving some gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (three sentences) and front-loaded: it first states the action and return values, then gives usage guidelines, then a warning. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool (1 param, no output schema, no annotations), the description covers the tool's purpose, return values, behavior, and important usage notes (time-limited URL, file_key usage). It is sufficiently complete for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides 0% coverage (no descriptions for parameters), so the description must compensate. It explains that file_path is a local file path on the machine, adding critical meaning beyond the schema's type 'string'. This fully clarifies the parameter's semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb 'Uploads' and clearly defines the resource 'Test Manager attachment storage'. It distinguishes this tool from siblings by being the only upload tool, and it clarifies the return values and their usage (file_key vs URL).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the output (file_key with other tools like tm.update_testCaseInstanceStep) and warns against speculative calls. It provides clear context for using the returned values, which is essential for an upload tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description carries full transparency burden. It fully discloses internal behavior: it fetches current test cases and PUTs a merged list, making repeated calls safe. It warns about the severe danger of invalid environment_id causing server errors until repaired. It also clarifies the upsert behavior and the type compatibility check.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is relatively long but highly informative. The first sentence captures the core purpose, and subsequent sentences add essential details in a logical order. Some redundancy (e.g., repeated emphasis on environment_id danger) could be trimmed, but it is not excessive given the importance. Overall, it earns its length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, but the description adequately covers what the tool does and its side effects. It explains the upsert behavior, the internal API mechanism, the danger of invalid environment_id, and the type matching requirement. This is sufficient for an action tool without a direct return value beyond the API response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description completely compensates. It explains each parameter: test_cases is an array of objects with required test_case_id, optional environment_id (with danger and source guidance), assignee (with lookup hint), and priority. It clarifies that environment_id can be omitted (defaults to 'No config selected'), and that multiple entries with different environment_ids are allowed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it adds or updates test cases in a test run, without removing existing ones. It specifies the optional fields (environment_id, assignee, priority) and explicitly distinguishes the upsert behavior. While sibling tools exist (e.g., bulkUpdate_testCaseInstances), the description's specificity on add vs. update and the unique danger of environment_id corruption make the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: it is for adding test cases, but also for reassigning or reprioritizing existing ones. It warns against speculative calls, explains the requirement for matching test case type to run type, and gives precise sources for environment_id (tm.get_environments or tm.get_testRunById). It also tells when not to use it (when you want to remove test cases).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses critical behavioral traits: including status resets all steps to 'Skipped', environment_id and remarks are silently ignored, and that the action is real and persistent. No annotations are provided, so the description fully bears this burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but each sentence adds value. It is front-loaded with purpose and usage, then covers constraints and dangers. Minor redundancy in the danger warning could be tightened, but overall well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex tool with no output schema and nested objects, the description covers what the tool does, when to use it, parameter details, side effects (step reset), and unsupported fields. It even warns against speculative calls. This is highly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage in the description, it adds significant meaning: explains the id comes from tm.get_testCaseInstancesByTestRunId's 'Instance ID' field, status values are enums, assignee is a user ID from tm.get_organizationUsers, and each instance needs at least one of status or assignee. This goes well beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates multiple test case instances in a single call, setting status and/or assignee per instance. It distinguishes from sibling tools like tm.update_testCaseInstance by noting that environment_id and remarks are not supported here.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly recommends using this tool over calling tm.update_testCaseInstance repeatedly for batch updates. It also tells when not to use it (for environment_id/remarks) and directs to tm.update_testCaseInstance instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully discloses behavioral traits: it is read-only, returns empty results for passed/never-executed/invalid IDs (not an error), and details the output structure including RCA fields and associated IDs. It also covers batch fetching behavior and pagination.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear flow: purpose, input parameters, output details, and important caveats. It is somewhat lengthy but each sentence adds value given the tool's complexity. Minor redundancy (e.g., mentioning batch fetching in two places) could be trimmed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is comprehensive, covering inputs, outputs, edge cases, prerequisites, and relationships to other tools. Despite no output schema, it explains the return structure in detail. It handles the complexity of multiple ID types and pagination thoroughly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero descriptions for its 6 parameters, but the description compensates fully by explaining each parameter (test_ids, job_ids, task_ids, stage_ids, page, limit) in detail, including their meaning, required combinations (at least one of the four ID arrays), and how they relate to other tool outputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves AI-generated root cause analysis for test executions, specifying the verb and resource. It distinguishes itself from sibling tools like tm.generate_testExecutionRCA by indicating it retrieves rather than generates, and from other get tools by focusing on RCA.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use this tool (for failed executions), warns against using it for passed or never-executed ones, and provides detailed guidance on how to identify suitable executions (e.g., checking status FAILED and non-empty Automation Test ID). It also explains the different ID types and their contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses read-only nature, describes edge case behavior (empty scope returns all-zero counts with explanatory message), and explains include_detail behavior. Without annotations, this fully covers behavioral aspects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is detailed but each sentence contributes value. It front-loads purpose and uses paragraph breaks for clarity. Could be slightly more concise, but structure is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 7 parameters, no output schema, and no annotations, the description covers all key aspects: purpose, usage, parameters, behavior, edge cases, and distinctions from siblings. No evident gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema coverage, the description explains scope parameters (types, caps, at-least-one requirement), include_detail (hydrate vs brief), and limit/offset pagination. Adds essential meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description specifies the tool returns a progress summary and paginated RCA results, clearly distinguishing it as the polling counterpart to tm.generate_testExecutionRCA. It mentions specific counts and scope, avoiding ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use: after calling tm.generate_testExecutionRCA for asynchronous polling. It notes the same scope as sibling tools, warns it does not trigger generation, and highlights pagination differences from tm.get_testExecutionRCA.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses resource consumption, template behavior (returns different ID), and filtering based on is_auteur_generated, with no annotations to cover.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Structured with clear intro, parameter list, and warnings, but slightly verbose; still well-organized and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 25 parameters, no output schema, and complexity, it covers all parameters, execution behavior, and warnings comprehensively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 25 parameters are described in prose with defaults, enums, and constraints (e.g., mutually exclusive group), compensating for 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it dispatches test cases to HyperExecute for real execution, distinguishing from preparation tools like tm.create_testRun and tm.add_testCasesToTestRun.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises not to call speculatively, suggests confirming with tm.get_testRunById, and explains the template behavior and execution constraints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that the operation fails if the issue is not actually linked, emphasizes that unlinking is a real, persistent action, and notes the difference in how the issue key is passed compared to the link operation. This fully exposes behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences: first covers purpose and reverse operation, second covers usage details and warnings. Every sentence adds value without any wasted words. Front-loaded with the core action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, no annotations, and simple string parameters, the description fully covers what the tool does, how it differs from siblings, required inputs, input formatting nuance, failure condition, and a caution. No gaps remain for an agent to select and invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so description must compensate. It explains that jira_issue_id is passed as-is (no prefix) and that all three parameters are required. While not providing detailed types or formats, the parameters are simple strings with minLength 1, so the added context about input format and failure condition is sufficient for an informed call.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it removes the link between a Jira issue and a Test Manager test case, explicitly names the reverse operation tm.link_jiraIssue, and distinguishes itself by noting the difference in passing the issue key. This provides a specific verb+resource and differentiates from its sibling tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explicitly lists required parameters (project_id, test_case_id, jira_issue_key), contrasts with tm.link_jiraIssue on input format (no org_id prefix needed), and warns against speculative calls because it is a persistent action. It gives clear context for when to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Despite no annotations, the description discloses key behaviors: attachments replace the whole list while omitted leaves existing untouched; new_steps are appended only; internally fetches snapshot_id; requires at least one change. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Every sentence adds value, no redundancy. Front-loaded with main action, then specifics. Efficiently covers constraints and behaviors.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 12 parameters, no schema coverage, no output schema, and no annotations, the description is thorough. It explains replace vs merge, append-only steps, snapshot fetch, and constraints. No obvious gaps for agent decision-making.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, requiring description to compensate. It explains most parameters (title, description, priority, etc.) and details attachment replacement and step append behavior. Missing explanation for commit_message, but overall adds significant meaning beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates metadata (listing fields) and/or appends new steps. This distinguishes it from siblings like tm.update_testCaseInstance which update instance steps, and tm.update_testCaseInstanceStep which modifies existing steps in an instance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly states when to use (update metadata or append steps), what not to do (cannot modify/delete existing steps, BDD, dynamic fields), and warns against speculative calls. Also mentions prerequisite to use tm.upload_attachment for file keys.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description fully discloses all behavioral traits: that status updates do not roll up, attachment_urls must be file keys (raw URLs silently fail), requires at least one field to change, and it's a persistent action. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is thorough and front-loaded with the main purpose, then detailed usage and warnings. Slightly verbose due to necessary nuance, but every sentence is informative. Slightly longer than minimal, but justified.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity, lack of output schema, and four parameters, the description covers identification, field semantics, side effects, prerequisites (upload attachment), and behavioral nuances. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates fully: explains the status enum, remarks as string, attachment_urls as array of file keys (with nuance), and test_run_step_id as string. Each parameter's meaning and constraints are clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it updates one step's status, remarks, and/or attachment_urls within a single test case instance, identified by test_run_step_id. It distinguishes from related tools like tm.update_testCaseInstance by noting that step status updates do not roll up to the instance's overall result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance on when to use (updating a step), how to get the step ID (from get_testCaseInstanceById), and what fields can be changed. It also warns not to expect automatic rollup and not to call speculatively, providing clear when-not-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![TestMu-AI-Test-Manager-MCP MCP server](https://glama.ai/mcp/servers/prakhar-gahlot/TestMu-AI-Test-Manager-MCP/badges/card.svg)](https://glama.ai/mcp/servers/prakhar-gahlot/TestMu-AI-Test-Manager-MCP)

Score Badge

Copy to your README.md:

[![TestMu-AI-Test-Manager-MCP MCP server](https://glama.ai/mcp/servers/prakhar-gahlot/TestMu-AI-Test-Manager-MCP/badges/score.svg)](https://glama.ai/mcp/servers/prakhar-gahlot/TestMu-AI-Test-Manager-MCP)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/prakhar-gahlot/TestMu-AI-Test-Manager-MCP'

If you have feedback or need assistance with the MCP directory API, please join our Discord server