SmartBear MCP server

Official

by SmartBear

Overview Schema Related Servers Score Discussions

TypeScript

Hybrid

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.37.0
Disambiguation3/5
Many tools have clearly distinct purposes (e.g., create_release, create_cycle, update_cycle, create_test_case), but there are multiple overlapping fetch tools for test cases, such as fetch_test_cases, fetch_test_case_details, fetch_test_case_version_details, fetch_test_case_steps, and fetch_test_case_steps_with_udf. While descriptions are detailed, the sheer number of similar 'fetch test case' tools could lead to misselection, especially for agents not familiar with QMetry's data model.
Naming Consistency2/5
Tool names are somewhat consistent with a 'qmetry_<verb>_<entity>' pattern, but there are notable deviations: 'qmetry_requirements_linked_test_cases_to_test_suite' and 'qmetry_link_platforms_to_test_suite' are verbose and inconsistent, while 'qmetry_fetch_test_case_runs_by_test_suite_run' is long. The mix of 'fetch' and 'link' verbs is predictable, but the naming is not clean and uniform across the board.
Tool Count1/5
With 52 tools, the server is far beyond the typical well-scoped 3-15 tool range. The sheer number is overwhelming and suggests that many sub-operations are exposed as separate tools rather than being consolidated. This makes the server hard to navigate and likely slows down an agent's decision-making.
Completeness4/5
The tool surface covers a comprehensive workflow: project management, requirements, test cases (CRUD, steps, versions, executions), test suites (CRUD, linking, executions), issues (CRUD, linking), automation import, quality gates, and UDF management. There are minor gaps (e.g., no explicit tool to delete a test case or test suite, and update/test suite operations are covered), but for the stated domain, the coverage is quite thorough.
Average 4.5/5 across 52 of 52 tools scored. Lowest: 3.6/5.
See the Tool Scores section below for per-tool breakdowns.
- 15 of 23 community issues answered or closed in the last 6 months
- 127 commits in the last 12 weeks
- Last stable release on August 20, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint, idempotentHint, and destructiveHint, so the safety profile is clear. The description adds valuable behavioral context about automatic resolution of viewId and folderPath, and it explains that the operation only reads data. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely long and repetitive. The same information—such as viewId auto-resolution, filter format, and projectKey consistency—is repeated across the Parameters, Use Cases, Examples, and Hints sections. This repetition adds no value and makes the description harder to parse, despite being well-structured in sections.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (17 parameters, no output schema), the description is quite complete. It covers pagination, filtering, sorting, and provides 17 diverse examples. It explains automatic behaviors and gives clear output expectations ('data' array). The only minor gap is a lack of detailed structure for the output objects, but it compensates for the absent output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions cover all 17 parameters, but they are generic for filter and sort (e.g., 'Filter criteria as JSON string'). The description adds substantial meaning by specifying exact JSON formats, providing valid field lists (e.g., entityKeyId, requirementStateAlias), and including numerous filter/sort examples. It also clarifies the auto-resolution behavior for viewId and folderPath, going beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch QMetry requirements' as a specific verb+resource action and highlights the automatic viewId resolution, which differentiates it from simpler list/read tools. However, it does not explicitly contrast with sibling tools like qmetry_fetch_requirement_details, so it loses a point for not drawing explicit lines against alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides extensive use cases, examples, and workflow hints, including the critical projectKey consistency guidance. Yet it does not explicitly state when to prefer this tool over alternatives (e.g., when you need a list vs. a single requirement detail), so the when-to-use guidance is implicit rather than explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, destructiveHint=false, and idempotentHint=true, so the safety profile is covered. The description adds that the tool generates a downloadable report file and that the backend returns it, but it does not disclose authentication needs, storage behavior, or response format. This is adequate but not rich.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than necessary because it repeats schema parameter details, but it is well organized with Toolset, Parameters, Use Cases, Examples, and Hints sections. The front-loaded purpose sentence and concrete example add value, so the structure earns its place despite some redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a straightforward export tool with four simple parameters and no output schema, the description covers the key workflow, required inputs, expected output, and a concrete example. It could mention download URL or file format details, but the provided information is sufficient for an agent to invoke it correctly in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all four parameters. The description repeats parameter definitions and adds minor clarifications like 'full HTML string' and 'without extension,' but these largely mirror the schema. It does not meaningfully compensate beyond the structured data.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Export HTML content as a downloadable report file via the backend.' This clearly distinguishes the tool from sibling QMetry tools, none of which are dedicated to exporting HTML reports. The title and use cases reinforce the purpose without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context through explicit use cases and a hint that this is 'typically called after generating report content from Execute Quality Gate Report results.' It does not explicitly name alternatives or exclusions, but the workflow placement is enough to guide selection among the many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, destructiveHint, and idempotentHint, so the description doesn't need to repeat safety. The description adds the output behavior: 'Returns a JSON object with full test suite details including UDFTypeData map and all UDF field values'. This clarifies what the caller receives but doesn't disclose additional behavioral traits like pagination, auth, or error handling. Given the annotations already cover the safety profile, the description provides adequate but not rich additional context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with sections: Toolset, Parameters, Output Description, Use Cases, and an Example. It is longer than necessary but every section adds value—especially the example and use cases. The main purpose is stated first, making it front-loaded. It could be trimmed of redundant parameter repetition, but the structure is clear and scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple fetch-by-id operation with a single required parameter, the description provides sufficient context: it explains the output shape (full suite details with UDF fields), gives a concrete example, and outlines common use cases. It doesn't describe error handling or edge cases, but for a read-only, idempotent operation with annotations covering safety, this is adequate. The absence of an output schema is compensated by the output description.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with descriptions (e.g., 'Test Suite ID (numeric ID)', 'Scope of the operation...'). The description repeats these exactly, adding no extra meaning beyond the schema. The description does tie parameters to use cases (e.g., id for specific suite) but that's implied. Since schema coverage is complete, the baseline is 3, and the description does not elevate it further.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches full detail data for a QMetry test suite including UDF field values. It quickly differentiates from sibling tools like qmetry_fetch_test_suites (which lists suites) and qmetry_update_test_suite (which updates), by specifying it retrieves full details for a single suite. The verb 'Fetch' plus the specific resource 'test suite details' leaves no ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Use Cases' section explicitly states when to use this tool: to get UDF field values, retrieve full metadata, or inspect before updating UDF values. This provides clear context for selection. While it doesn't mention alternatives or when not to use it, the use cases implicitly distinguish it from listing or updating tools. The lack of explicit exclusions is a minor gap.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true and idempotentHint=true, but the description describes a mutating operation: importing results creates/updates test suites, test cases, and execution results, and mentions folder creation and field updates. This contradicts the read-only annotation, creating an annotation contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely long but well-structured with clear sections (Parameters, Use Cases, Examples, Hints, Workflow). It contains some redundancy (e.g., double numbering, repeated reminders), but all sections provide useful operational context that earns their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 17-parameter tool with no output schema, the description covers file requirements, entity types, hierarchy mappings, optional params, dependencies, field behavior, error handling, workflow, and expected outputs. It is exhaustive and would allow an agent to use the tool without needing external documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description's 'Parameters' and 'Hints' sections add extensive meaning beyond the schema: dependencies (cycleID requires releaseID and projectID), automationHierarchy behavior per framework, custom field formats, lookup-list behavior, error handling, and multiple examples. Even with full schema coverage, it enriches parameter semantics significantly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Import/Publish automation test results from TestNG, JUnit, Cucumber, Robot, HPUFT, or QAF frameworks into QMetry', clearly naming the verb, resource, and scope. It distinguishes this tool from sibling fetch/update/link tools, which all have different purposes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Workflow' section provides a step-by-step process and the 'Use Cases' list specific scenarios (e.g., 'Import TestNG XML results after CI/CD pipeline execution'). However, it does not explicitly name alternative tools for cases not suited to import (e.g., manual test case creation), relying on the implied distinction from siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only/idempotent/non-destructive behavior, so the description only needs to add operational detail. It does add pagination defaults, filter AND-logic, and the tsID naming caveat, but it also contains misleading text: getLinked is described as 'issues linked with this Test case Run' rather than test cases linked to a suite, and Example 4 claims '50 items per page' with limit=25. These inconsistencies reduce trust.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is heavily over-sized: 10 use-case bullets and 22 hints contain substantial redundancy, such as repeating the getLinked default and tsID retrieval instructions multiple times. It is structured and front-loaded, but many sentences do not earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description compensates with an output description, examples, pagination guidance, filter semantics, and ID lookup instructions. Minor factual inconsistencies and the overlong format prevent a perfect score, but it is still operationally complete enough for correct tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, giving a baseline of 3, but the description adds substantial meaning: exact steps to retrieve tsID, warnings against wrong parameter names, filter field/ID mappings, priority/status/enum conventions, and concrete JSON examples. This goes well beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line, 'Get test cases that are linked (or not linked) to a specific test suite in QMetry', names a specific verb, resource, and scope. It clearly distinguishes this from siblings like qmetry_fetch_test_cases and qmetry_fetch_test_suite_details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The Use Cases section provides concrete functional scenarios, and the Hints section explains how to obtain tsID and when getLinked=false is useful for gap analysis. It does not explicitly name sibling tools as alternatives or exclusions, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a non-read, mutating operation; the description reinforces that with 'Link' and gives an expected output shape. It adds some useful context like the default behavior of fromReqs and the prerequisite API calls, but it does not disclose potential side effects, idempotency behavior, or what happens when understanding selected test cases are already linked.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with headers and front-loaded with the first sentence, but it is long. The 'Parameters' section repeats content already present in the input schema, and the three examples are highly redundant, mostly changing only IDs and expected test-case names.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description still provides both a general 'JSON object with linkage status and details' and worked examples that show concrete expected outcomes. It also supplies prerequisite APIs and hints, so the agent can compose a successful invocation. It falls slightly short of fully complete because the output object's exact shape and failure modes are not specified.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description goes further by explaining how to obtain tsID, how to discover tcvdIDs from a requirement entityKey, and giving concrete example values with expected linked test cases, which materially helps the agent construct valid arguments.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a precise action: 'Link test cases (including those linked to requirements) to a test suite in QMetry.' This clearly identifies the verb, resource, and domain, and the 'requirements linked' framing plus the fromReqs flag distinguishes it from the sibling qmetry_link_test_cases_to_test_suite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit use cases, practical hints for resolving prerequisite ids, and guidance on when to set fromReqs to true. It does not explicitly name the direct-link sibling as an alternative or state when not to use this tool, but the workflow hints are strong enough for correct selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already convey idempotency, non-read-only, and non-destructive behavior. The description adds valuable behavioral context by explaining that the project context persists for the current session, that this operation must precede test case operations for non-default projects, and that output includes confirmation and project metadata. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections, use cases, examples, and hints, but it is longer than necessary. The three examples are nearly identical and mostly repeat the same invocation with different string values, and the parameter listing duplicates the schema. Still, it is front-loaded and organized enough to remain usable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-optional-parameter context-setting tool, the description covers preconditions, use cases, output shape, session persistence, and common project key values. It does not describe error behavior for invalid project keys, but hints to check with the QMetry admin and includes a validation use case, which is adequate for most selection and invocation needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already covers projectKey fully with type, default, and description. The description mostly repeats the schema but adds examples, common project key names, and practical hints. This adds slight value but does not substantially go beyond what the schema provides, so the baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Set current QMetry project for your account', which clearly names the action (set) and resource (current QMetry project). It is clearly distinguishable from sibling fetch/list tools like qmetry_fetch_qmetry_project_info and qmetry_fetch_qmetry_list_projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The Use Cases and Hints provide strong context for when to call this tool, especially before test case operations and for establishing session project context. It does not explicitly name alternatives or say 'use fetch tools when you only need to view project info', but the set-vs-fetch distinction is clear from the sibling names and examples.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already mark it read-only/idempotent/non-destructive, and the description adds concrete archive-filtering behavior: 'Default behavior always excludes archived items unless explicitly requested' and the three payload examples. It also discloses the output as a JSON project hierarchy and clarifies the default for showArchive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The content is well-structured with headings and bullets, but it is redundant: Examples 1 and 2 describe the same false/default behavior, and Hints 3–6 restate the same payload scenarios. The odd numbering in Hints ('2. PAYLOAD SCENARIOS:' followed by 3-6) further hurts clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter, read-only fetch with no output schema, the description is comprehensive: it defines parameters, provides three examples, and explains the release-cycle hierarchy and default behavior. It does not discuss pagination or error cases, but these are less critical for a simple read tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema covers both parameters fully, so baseline is 3; the description adds value by explaining that omitting showArchive results in showArchive:false, and that projectKey defaults to 'default'. However, most parameter semantics already exist in the schema, so the bonus is modest.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Fetch QMetry releases and cycles from the current project', giving a clear verb and resource. It is unambiguous versus sibling fetch tools because it names releases and cycles, though it does not explicitly contrast itself with related fetch tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Use Cases list several reasons ('test planning', 'reporting', 'search for specific releases using release name or ID') and Hints explain the release-cycle hierarchy for planning. It gives clear operational context but never states when not to use it or recommends an alternative sibling tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already specify readOnlyHint: false and destructiveHint: false, and the description's 'link' action is consistent with a mutating but non-destructive operation. The description adds some context about output being a JSON linkage status object, but it does not explain error behavior, idempotency, or side effects beyond the basic link operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections for Parameters, Output, Use Cases, Examples, and Hints, and the core purpose is front-loaded. It is somewhat verbose and a few Use Case bullets are close to restatements, but the overall organization makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a mutation tool with no output schema, so the description must compensate for missing return-value clarity. It provides a high-level output description, examples, dependency hints, and required-parameter emphasis, but exact response fields and error cases are not described.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already documents all 4 parameters with 100% coverage, so the baseline is 3. The description adds value with the critical tcrId naming caveat and concrete JSON examples showing exactly how to pass issueIds and tcrId, which improves correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Link one or more issues to a QMetry Testcase Run', a specific verb+resource statement that clearly identifies the tool's behavior. It also distinguishes itself from sibling linking tools like qmetry_link_requirements_to_testcase and qmetry_link_platforms_to_test_suite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The Use Cases and Hints sections provide strong contextual guidance, including when to link single vs. multiple issues and which other tools to use to obtain issueIds and tcrId. It lacks an explicit 'when not to use this tool' or comparison with alternative sibling tools, so it doesn't earn a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations include readOnlyHint:false, destructiveHint:false, idempotentHint:false. The description says 'Link test cases,' which is a write operation consistent with readOnlyHint:false. It adds context about the output (JSON object with linkage status) and expected results in examples. However, it does not disclose potential side effects (e.g., behavior on duplicate links) or discuss idempotency, relying on annotations to convey non-idempotency. No contradiction exists, but the description could add more behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections (Parameters, Use Cases, Examples, Hints), but it is verbose. Three examples with similar content add redundancy; a single example would suffice. The parameter list repeats schema descriptions, adding length. However, the critical naming warnings and hints are valuable. The structure is logical but not concise enough for optimal quick scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has 3 parameters, no output schema, and moderate complexity. The description provides a complete picture: parameter details, output description, use cases, examples with expected outputs, and acquisition hints. It fully compensates for the lack of an output schema by describing the JSON return. The hints ensure the agent can correctly obtain and supply parameter values, making the tool's usage self-contained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%—the schema already describes all parameters in detail. The description adds value by emphasizing CRITICAL parameter naming constraints ('do NOT use testSuiteId...') and providing hints on how to derive values for tsID and tcvdIDs from other APIs. It also explains the fromReqs option's default and purpose. While the schema covers semantics, the description enhances usability with practical examples and acquisition guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Link test cases to a test suite in QMetry.' The verb 'link' and resources 'test cases' and 'test suite' are specific. It differentiates from siblings like qmetry_link_requirements_to_testcase and qmetry_link_platforms_to_test_suite by specifying the target entity. The use cases list distinct scenarios, reinforcing the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Hints' section provides explicit guidance on when to use the tool and how to obtain required parameters (tsID via Fetch Test Suites for Test Case API, tcvdIDs via Testcase/Fetch Versions API). It also clarifies when to set fromReqs to false. However, it does not explicitly compare with sibling tools or state when NOT to use it, such as alternative linking tools. The use cases imply usage context but lack explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond annotations by explaining the JSON output, date-format ambiguity, auto-resolution of projectID from projectKey, defaults for cycle.isLocked/isArchived, and recommends FETCH_RELEASES_CYCLES for verification. No statement contradicts the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with headings and a clear opening sentence, but it is verbose: 17 hints and 4 examples contain redundancies (e.g., naming conventions, release hierarchy, post-create association). It earns points for structure, but not for conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Even without an output schema, the Output Description covers return value shape, and examples show expected outputs. Hints cover required fields, date formats, optional cycle behavior, and post-creation verification, making the tool actionable in context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The Parameters section mostly mirrors the schema, but hints add meaning: release.name is required, date format is instance-dependent, projectID auto-resolves, and cycle defaults are false. Examples illustrate valid JSON shapes for release and cycle, so the description compensates for the 75% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line 'Create a new release in QMetry with optional cycle for test planning and execution tracking' names a specific verb, resource, and optional nested scope. It clearly identifies release creation, but it does not explicitly contrast with the sibling qmetry_create_cycle tool when only a cycle is needed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Use Cases enumerate six relevant scenarios (e.g., major product versions, release hierarchies, sprint planning) and hint #7 says to omit cycle if only creating a release. However, there are no explicit exclusions or named alternatives, such as directing cycle-only requests to qmetry_create_cycle.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false, covering safety. Description adds a critical note about parameter naming but does not disclose other behavioral traits (e.g., error handling, rate limits). No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with clear sections (overview, parameters, use cases, examples). Front-loaded with the purpose, concise overall, every section earns its place without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple fetch tool with no output schema, it covers purpose, parameters, use cases, and an example. It hints at expected output ('Status, progress, and details') but lacks detailed return format. Given the simplicity and annotations, it is adequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. Description adds important context beyond schema: it emphasizes the exact parameter name 'requestID' and clarifies that it accepts a string or number (schema says number). Also provides an example JSON.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Fetches the status of an automation import job by request ID' with a specific verb and resource. It distinguishes from siblings like qmetry_import_automation_test_results (which triggers imports) and other fetch tools by focusing on status retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a 'Use Cases' section listing concrete scenarios (check completion, retrieve progress, monitor CI/CD). It implies the tool is used after an import is initiated, but does not explicitly mention alternatives or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already state readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the read-only nature is covered. The description adds valuable behavioral context beyond annotations by explaining showArchive behavior, pagination defaults, default projectKey, and filter field examples, giving the agent a strong sense of what the request does and what the response contains.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is highly repetitive: the same fields are listed in the opening sentence, the parameter section, use cases, and example output text. The parameter details largely duplicate the schema, and several examples are near-identical (e.g., showArchive false vs true). While it is front-loaded and well-sectioned, it is far from concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema and seven parameters including a nested object, the description is unusually complete. It covers the purpose, output fields, use cases, pagination, filtering, archive behavior, and provides multiple concrete JSON examples with expected outcomes. An agent has enough context to call this tool correctly without external documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema already documents all parameters at roughly 86% coverage, so the baseline is 3. The description adds extra semantics through examples and hints, especially the filter JSON format, common filter fields ('name', 'projectKey'), and the meaning of showArchive true/false, which goes beyond the raw schema property descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with a specific verb and resource: 'Fetch QMetry projects list' and enumerates the returned fields (projectID, name, projectKey, isArchived, viewIds, folderPath). It clearly distinguishes this list-oriented tool from sibling tools like qmetry_fetch_qmetry_project_info by emphasizing the list scope and the fields needed for downstream operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Use Cases' and 'Hints' sections provide explicit guidance on when to use the tool: fetching available projects, validating access, supporting pagination, and filtering. It lacks an explicit exclusion like 'use qmetry_fetch_qmetry_project_info for a single project,' but the contextual signals are clear enough for an agent to select this tool appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false, idempotentHint=false, and destructiveHint=false, so the description doesn't need to establish the mutation behavior. The description adds important behavioral context: the API silently discards poorly-formatted data (DATETIMEPICKER), the tool can return errors with recoverable exceptions (CO.MANDATORY_FIELDS_MISSING), and it emphasizes the need to auto-apply defaults to avoid data loss. These are beyond what annotations provide. It doesn't significantly contradict annotations — readOnly=false aligns with a create operation. Score 4 due to some redundancy from the repetitive sections.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely long (over 100 lines) and repetitive. Many sections are repeated — e.g., the pre-create check is described in both STEP 0 and MANDATORY PRE-CREATE CHECK, the error recovery is explained twice, and the date format check is mentioned multiple times. The front-loaded section initially lists parameters, then the description body repeats parameter explanations. While it is well-structured in parts (clear headers, decision matrix, examples), the excessive length hurts readability for an agent context hook. Many sentences could be merged to half the length without losing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a highly complex tool with 15 parameters, no output schema, and multiple hidden dependencies (Fetch UDF Layout, Fetch Project Info, Fetch Releases and Cycles). The description addresses all these dependencies, provides error recovery, date format, ID resolution, and even examples with plain-language translations. Despite being over-long, it is functionally complete for an agent to use correctly. No output schema exists, so the description covers return value hints ('Expected Output' in examples).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
While the input schema covers all 15 parameters (100% coverage), the description adds significant value: it explains the exact ID resolution for issueType/issuePriority/issueOwner/component from specific list objects, how to format affectedRelease/affectedCycles as arrays, the special handling for environment (no ID lookup), the udfFields format mapping to fieldTypeName, and the CASCADINGLIST structure. It also provides the default value routing logic (system fields to top-level, UDF to udfFields). This significantly exceeds what the schema provides, going beyond 3 mid-blocks to 5.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a defect/issue internally in QMetry, which is a specific verb+resource. It distinguishes itself from siblings like qmetry_fetch_defects_or_issues and qmetry_update_issue, though it doesn't explicitly name alternatives in the description. The 'Toolset: Issues' heading and detailed parameter list make the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description excels here with a detailed pre-create check workflow, error recovery procedures, date format handling, release/cycle resolution instructions, and a full parameter resolution table. It explicitly states when to call 'Fetch UDF Layout' before create, how to handle mandatory/default fields, and provides a decision matrix for when to ask user vs auto-fill. This is top-tier usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, which are well complemented by the description. The description adds that it returns gate configuration and mentions error behavior if AI Agent is not enabled, providing useful context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections for parameters, use cases, examples, and hints. It is longer than usual but each section adds value; however, there is some redundancy with the schema that could be trimmed for brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema, the description provides expected output (gate configuration object with assessment scope, gate criteria, thresholds). It also notes prerequisites and error conditions, covering key aspects for a fetch operation. Lacks details on output structure but adequate for a configuration fetch.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are documented in the schema. The description reinforces key distinctions (projectId vs projectKey) and emphasizes required parameters, adding value beyond the schema, though much is redundant.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it fetches quality gate configuration for a project and AI agent, including assessment scope and gate criteria. It distinguishes from siblings by focusing on configuration retrieval vs. execution (qmetry_execute_quality_gate_report) and other fetch operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear use cases (e.g., retrieve criteria before generating a report) and hints that it should be called before 'Execute Quality Gate Report'. It doesn't explicitly state when not to use it, but the context is sufficient for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations provide readOnly=false, idempotent=false, destructive=false; the description adds substantial behavioral context beyond these: status IDs are project-specific and must be fetched dynamically or 'tests will be updated with incorrect status', the tool is intended to be invoked multiple times in sequence for multi-execution updates, isBulkOperation is auto-detected from comma-separated IDs, and username/password become required under Part 11 compliance. No contradiction with annotations exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is 116 lines with severe redundancy: the runStatusID-fetching process is explained at least four times (parameter descriptions, hints 3-10, detailed process 40-55, workflow 85-98), and the common status names list appears twice (lines 17-23 and 79-84). The closing lines (113-116) repeat platitudes about being 'essential' and 'critical' without new information. While it is well-sectioned and front-loaded, too many sentences fail to earn their place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 14-parameter mutation tool with no output schema, the description is remarkably complete: it covers prerequisite fetch workflows for entityIDs, qmTsRunId, runStatusID, and dropID; conditional auth; entity type semantics; expected outputs via the Expected Output lines in each example; and the multi-call workflow for full-suite updates. There are no significant missing operational details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds genuine value beyond the schema: seven worked JSON examples showing parameter combinations, field-mapping critical notes (entityIDs must be comma-separated string, qmTsRunId string, runStatusID numeric), the auto-detection default for isBulkOperation, and the common status name list. Some parameter details are duplicated verbatim from the schema, but the examples and type-mapping notes elevate beyond baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line uses a specific verb+resource+scope: 'Update execution status for individual or multiple test case runs in bulk.' The API endpoint (PUT /rest/execution/runstatus/bulkupdate) and toolset classification further anchor it. It is clearly distinguished from siblings like qmetry_bulk_update_test_run_udfs (updates UDFs, not status) and qmetry_import_automation_test_results (imports results rather than updating status).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Extensive context for when to use: manual execution updates, automated test result integration, compliance/audit trails, and the workflow section maps user prompts ('if user says execute status to not run...') to concrete tool invocations. It also specifies conditional usage (Part 11 compliance auth) and when multi-call sequencing is required for ALL executions. However, it never explicitly names alternatives to use instead (e.g., qmetry_import_automation_test_results for bulk automation imports), so it lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds valuable behavioral context beyond annotations: output shape ('JSON object with builds list and pagination metadata'), pagination defaults, filter behavior, and the QMetry terminology note that builds are also known as 'drops.' No contradiction with annotations exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than average but well-structured with clear sections: description, parameters, output, use cases, examples, and hints. The examples earn their place by demonstrating non-obvious filter syntax. Minor redundancy exists because the Parameters section largely duplicates the input schema, but overall the organization keeps it usable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 6 parameters and no output schema, the description compensates thoroughly: it describes the output format, provides four worked examples, explains pagination defaults, and gives filter field hints. It covers the main usage scenarios and edge cases like empty payloads and archive-status filtering, making the tool fully understandable without external documentation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by providing concrete filter examples for name and archive status, explaining the JSON string format, listing common filter fields, and showing that an empty payload {} triggers default behavior. This is genuinely helpful for correctly constructing filter and pagination parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Fetch QMetry builds from the current project.' This clearly identifies the operation and scope, and the resource 'builds' is distinct from sibling tools like releases, cycles, test cases, and requirements. The title and description align without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Use Cases' section provides clear contexts such as test execution planning, reporting, CI/CD integration, and filtering builds. It also gives practical hints about defaults and filter fields. However, it does not explicitly state when NOT to use this tool or mention alternative tools, so it stops short of full exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already establish readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so no side-effect disclosure is needed. The description adds useful behavioral context beyond annotations: response contains a UDFTypeData map, the defectId must come from the list response's 'id' field, and entity keys require a pre-fetch step. There is no contradiction with the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections: summary, parameters, output, use cases, examples, and hints. It is longer than necessary because the Parameters section mostly repeats input schema descriptions, but the additional examples and workflow guidance are valuable. No critical content is missing, though minor redundancy prevents a perfect score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given there is no output schema, the description compensates well by stating that the response is a JSON object with a 'data' property containing full issue details and the UDFTypeData map. It also explains how to obtain the required defectId, provides expected outputs, and gives a complete workflow for fetching UDF values. This is sufficient for a moderately complex read-only tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description goes beyond the schema by warning not to derive defectId from the entity key suffix and by describing an AUTO-RESOLVE flow when a user supplies an entity key instead of a numeric ID. The examples also reinforce the expected parameter usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Fetch') plus a concrete resource ('full detail data for a QMetry issue including UDF field values'), which clearly distinguishes this tool from list-oriented siblings like qmetry_fetch_defects_or_issues. It also states the output focus on UDFTypeData and custom fields, making the tool's purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit 'Use Cases' and a detailed 'Workflow' explaining exactly when to call this tool, including how to resolve an entity key via Fetch Issues/Defects first. It does not explicitly name excluded alternatives or say 'do not use this for listing issues', but the context is clear enough for correct selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the core read-only safety profile is covered. The description adds meaningful behavioral guidance: NEVER use user-provided IDs directly as entityId, ALWAYS fetch execution data first, and the entityId naming trap. No contradiction with annotations was found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is clearly sectioned and front-loaded, but it is excessively verbose: ten use-case bullets and dozens of hint lines repeatedly re-explain entityId resolution, getLinked defaults, and pagination. The structure helps, but many sentences are redundant and not every line earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 10-parameter tool with no output schema, the description is unusually complete. It covers the required ID provenance, dependent execution APIs, linked-vs-unlinked behavior, extensive filter guidance, six examples, pagination, and expected response content. An agent has enough information to invoke this tool correctly without additional discovery.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description adds substantial meaning beyond the schema: it explains how to derive entityId from data[<index>].tcRunID, documents the filter object structure, enumerates supported filter fields, maps type/state/priority IDs, and walks through rich examples. This is far more useful than the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description immediately states a clear verb-and-resource: get issues linked (or not linked) to a specific test case run in QMetry. This distinguishes it from siblings such as qmetry_fetch_issues_linked_to_test_case and qmetry_link_issues_to_testcase_run by focusing on the test case run entity and the getLinked toggle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides broad usage context via Use Cases and detailed workflow hints for when to invoke the tool, including gap analysis with getLinked=false, defect tracking, traceability audits, and the required tcRunID resolution process. It does not explicitly name alternative tools for when-not-to-use, but the guidance is otherwise clear and actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are readOnly, idempotent, non-destructive; the description agrees and adds substantial behavioral context. It reveals the endpoint (`viewColumns`), automatic `viewId` resolution, output shape including `filterTemplate` and `totalCount`, and details about how UDF values appear in responses, including field naming conventions and lookup/cascading list behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with headings, examples, and hints, making it easy to scan. However, it is lengthy and the Parameters section largely repeats info already present in the input schema, adding redundancy. The behavior-specific hints and output details justify most of the remaining length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries full responsibility for explaining behavior and return values. It does so thoroughly via 'Output Description,' a concrete example, and seven detailed hints covering `filterTemplate`, UDF display values, numeric IDs, and lookup/cascading list representations. This is more than enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already includes the same detailed explanations, including the `tcID` naming caveat and `viewId` auto-resolution. The description's 'Parameters' section largely duplicates the schema, adding little beyond what structured fields already provide. It does include examples, but these do not add new parameter meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb-resource pair, 'Fetch test case steps including UDF field values via viewColumns endpoint,' which clearly states the tool's scope. The name and title also distinguish it from siblings like `qmetry_fetch_test_case_steps` by emphasizing UDF values, so there is little ambiguity about purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Use Cases' section explicitly lists scenarios: getting step UDF values, retrieving steps before updating step UDFs, and inspecting step-level UDF data for reporting. This gives clear context for when to use the tool, though it does not explicitly mention an alternative tool or state when not to use this one.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations include readOnlyHint=false, destructiveHint=false, and idempotentHint=false, which already indicate a non-read, non-destructive, non-idempotent operation. The description adds context by stating it links requirements (implying modification) and provides output description ('JSON object with success status and linkage details'). It does not contradict annotations, but it does not disclose side effects like whether existing links are replaced or appended, or if there are permissions required. Annotations carry the basic mutation info, so a 3 is appropriate; no contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is comprehensive and well-structured with sections for parameters, output, use cases, examples, and hints. It is longer than necessary, but each section earns its place: the hints are highly actionable, and the example is valuable. It is front-loaded with the core purpose, and the structure aids scanning. There is minor redundancy (parameter descriptions echoed from schema), but it's acceptable for usability. Not as concise as the TDQS 4.3 example, hence 4.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (3 parameters, no output schema, no nested objects), the description is complete. It includes all necessary information: purpose, parameter semantics, output description, use cases, and critical hints for resolving IDs, which are non-obvious and require external API calls. The annotations provide safety hints, so the description doesn't need to repeat them. This description fully equips the agent to use the tool correctly, surpassing the minimum viable bar.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, each parameter has a description in the schema. However, the tool description enhances parameter understanding significantly by providing examples (e.g., 'VT-TC-26', '5448515', '5009939,5009937,4970699') and detailed hints on how to obtain each parameter (e.g., 'To get the tcID, call the Testcase/Fetch List for Bulk Operation API and use data[<index>].entityKey'). This goes beyond the schema's basic descriptions, adding practical guidance. Baseline is 3 due to full schema coverage, but the extra detail justifies a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Link one or more requirements to a test case by entityKey and version IDs.' It specifies the action (linking), the resource (requirements to test case), and the identifiers used. It is distinct from sibling tools like qmetry_link_test_cases_to_test_suite, which links test cases to suites. The title and description are consistent, and the toolset label ('Requirements') helps contextualize it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit guidance on when to use this tool: 'Use Cases: 1. Link requirements to a test case for traceability 2. Bulk link multiple requirements to a single test case 3. Automate requirement coverage mapping.' It also mentions alternatives implicitly by naming the toolset and providing hints to resolve IDs via other APIs. However, it does not explicitly state when NOT to use it, but the use cases and hints are sufficient for differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are minimal (only readOnlyHint/idempotentHint/destructiveHint all false), so the description carries the full burden — and it delivers massively. It discloses the TC.VERSION_NOT_SYNCED error and automatic retry behavior, silent data-loss on wrong date formats, the anti-duplication tcStepID mechanism, default-values sweep behavior, and UDF dual-key requirement. This goes far beyond what annotations convey. No contradiction with the readOnlyHint=false annotation; the description's mutation semantics align.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness1/5
Is the description appropriately sized, front-loaded, and free of redundancy?
This is severely over-specified — roughly 400 lines covering the same workflows multiple times. Version-creation guidance, anti-duplication rules, and the TC.VERSION_NOT_SYNCED retry pattern are each explained 3+ times with near-identical content (e.g., the verified payload appears in both pseudo-code and JSON forms, and the incremental tcVersion rule is restated across sections). While headers provide some structure and the first sentence is front-loaded, this reads as a full product manual, not a tool description. Every-sentence-earns-its-place fails categorically.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the extreme complexity (27 params, nested step/removeSteps objects, dual-mode behavior, UDF handling, and no output schema), the description is remarkably complete: it provides an output description, 9 worked examples with expected outputs, error-handling patterns, and field-mapping guidance. This compensates for the absence of an output schema. The only minor gap is that some common fields (owner, component, description) rely on hints rather than dedicated param explanations, but overall completeness for this complex tool is high.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 59% schema coverage, the description compensates well for the tricky parameters: tcID's naming trap, tcVersionID vs tcVersion distinction, withVersion semantics, UDF/udfFields dual structure, and step handling with tcStepID. However, plain parameters like name, priority, component, owner, testingType, testCaseState lack descriptions both in the schema and in the parameters section — they're only explained indirectly through hints about project-info ID resolution. The critical params are well covered, but several simple ones are underspecified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line states the verb-resource-purpose crisply: 'Update an existing QMetry test case OR create a new version by tcID and tcVersionID, with auto-resolution from entityKey.' It immediately establishes the dual-mode nature (update vs. version creation) that distinguishes it from sibling fetch/create tools. The extensive parameter and example sections reinforce exactly what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives the most explicit when-to-use guidance possible: a dedicated 'CRITICAL - VERSION CREATION vs UPDATE DISTINCTION' section maps user phrasing to MODE 1 vs MODE 2, with a decision tree ('User says create new version → MODE 1'). It names sibling tools for prerequisites ('Call Fetch UDF Layout', 'Fetch QMetry Project Info', 'FETCH_TEST_CASE_STEPS') and what to do if ambiguous. This is near-textbook usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond the annotations, the description discloses important backend behavior: skipping Fetch UDF Layout can cause CO.MANDATORY_FIELDS_MISSING errors, wrong DATETIMEPICKER formats can cause silent value loss despite API success, and defaults must be swept to avoid missing data. It also provides an auto-recovery procedure, which goes well beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-sectioned and uses tables, but it is extremely repetitive: the STEPS DECISION RULE appears in the parameter explanation, the main description, the hints block, and the examples almost verbatim. It could be reduced by a substantial amount without losing information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex 14-parameter tool with nested objects, low schema coverage, and no output schema, the description is remarkably complete. It covers preconditions, folder resolution, ID lookup, date formatting, step handling, release/cycle mapping, UDF workflows, and output shape in one place.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema description coverage is only 36%, the description compensates thoroughly: tcFolderID auto-resolution, estimatedTime in seconds, udfFields type mappings, releaseCycleMapping version requirements, and the full steps inclusion rule are all documented. It also includes concrete prompts and payload examples showing how to map user language into valid parameter values.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific action: 'Create a new test case in QMetry with steps, metadata, and release/cycle mapping.' This immediately distinguishes it from sibling tools like create_test_suite or create_release. The name and title reinforce the same intent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description contains explicit decision tables, 'NEVER' conditions, and mandatory pre-create calls to Fetch UDF Layout and Fetch QMetry Project Info, which strongly guide when and how to invoke the tool. However, it does not explicitly contrast this tool with sibling qmetry_update_test_case for scenarios where updating an existing test case would be preferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, and the description adds significant behavioral context: automatic viewId resolution, consequences of using a stale viewId, filter conjunction logic, pagination defaults, and the meaning of isArchived values. It does not contradict the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is overly verbose and repetitive: tsID and viewId retrieval instructions appear both in the parameter list and again in Hints, and many lines are filler such as 'This tool is essential for test execution analysis and reporting'. It is structured with sections, but excessive repetition and an internal inconsistency in viewId field paths hurt readability and scanning.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 10 parameters, no output schema, and multiple upstream dependencies, the description is remarkably complete. It covers prerequisites, automatic resolution behavior, flexible filter format, pagination, and expected output shape, so an agent can call the tool correctly with minimal guessing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description still elevates parameter understanding by correcting likely parameter-name mistakes, explaining exactly how to obtain tsID and tsFolderID, describing filter fields such as releaseID, platformID and isAutomatedFlag, and providing complete JSON examples. The examples make the filter and pagination behavior concrete for an agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening sentence, 'Get executions for a given test suite in QMetry', clearly states a specific verb, resource, and scope. It distinguishes the tool from sibling fetch tools by emphasizing the 'by test suite' context, and the use cases further reinforce its role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit use cases, workflows, and dependencies such as using FETCH_PROJECT_INFO first, filtering with IDs from FETCH_RELEASES_AND_CYCLES, and paginating large result sets. It does not explicitly contrast this tool with very similar siblings like qmetry_fetch_test_case_runs_by_test_suite_run, so a clear 'when not to use' statement is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds valuable behavior details: the getLinked flag toggles linked vs. unlinked results, pagination behavior, filter capabilities with specific field names and value types, and linkage level distinctions. It also describes the output shape (issues array with details, priorities, status). This goes well beyond the annotations and schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely long (several paragraphs, bullet lists, examples, and 22 numbered hints) and repeats parameter details already in the schema. While well-structured with headers, it contains redundancy (e.g., parameter descriptions duplicated, repeated hints about getLinked default). It is not concise, though the first sentence is clear and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With 8 parameters and no output schema, the description compensates with an output description ('JSON object with issues array containing issue details, priorities, status, and linkage information') and extensive filtering/pagination guidance. It is complete enough for an agent to use effectively, though the output shape could be more explicit.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with already descriptive parameters. The description adds heavily: detailed tcID resolution hints (e.g., from MAC-TC-1684 to numeric ID), filter field enumerations (summary, executedVersion, linkageLevel, etc.), meaning of issue type/priority/status IDs, and pagination semantics. This extra context significantly aids parameter interpretation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear, specific statement: 'Get issues that are linked (or not linked) to a specific test case in QMetry.' This uses a specific verb-resource pair and immediately distinguishes from siblings like fetch_linked_issues_of_test_case_run by focusing on test cases rather than test runs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a dedicated 'Use Cases' section with 10 scenarios (defect tracking, gap analysis, traceability, etc.) that clearly indicate when to use this tool. However, it does not explicitly mention alternatives or when NOT to use it (e.g., when to use the test-case-run variant). The guidance is present but lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, covering the safety profile. The description adds meaningful behavioral context beyond annotations: the numeric ID vs entity key distinction, version semantics, and that the response includes complete requirement information and custom fields.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with clear sections and front-loaded purpose, but it is verbose. It duplicates the parameter schema exactly, includes a largely redundant list of use cases, and has a trivial example. The valuable hints could be preserved more concisely.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only fetch tool with no output schema, the description covers output expectations, required parameter behavior, entityKey resolution, version defaults, and when to use it over list tools. This is sufficient even without an output schema, and annotations cover safety traits.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description largely repeats schema text, but the Hints add operational value: explain that version 1 means latest unless specified, and how to obtain the numeric ID if an entity key is provided. This goes beyond mere schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line, 'Get detailed information for a specific QMetry requirement by numeric ID', clearly states the verb, resource, and scope. It also explicitly distinguishes this from list-type tools by noting that it provides details 'not available in the list view' and referencing sibling qmetry_fetch_requirements.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The Hints section gives explicit when-to-use and how-to-use guidance, including resolving entityKey via FETCH_REQUIREMENTS first, the required version parameter, and using this tool when detailed information is needed. It also clarifies the alternative path for entity keys, which helps an agent choose correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly=true, idempotent=true, destructive=false, and the description does not contradict these. The description enriches behavior beyond annotations by explaining that HTML is stripped from rich text UDF fields, that the response contains 'testRunUdfs' as a key-value map, that a 'hasTcRunUdf' flag indicates whether to attempt to read these fields, and how the system auto-resolves viewId. It also describes the forced output formatting (unified table) which is a requirement of response rendering. However, it does not explicitly state the tool's read-only nature (already covered by annotations), and includes a lot of instruction on output formatting that is not strictly about the tool's runtime behavior, thus leaving a slight gap about actual side effects (none).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely lengthy and contains substantial redundancy. The same instructions about tcrUdfFilter appear in the parameter description, in the 'Hints' section, and in the examples. The 'Hints' section also repeats the response format rules that are already in the 'Output Description'. The 'CRITICAL WORKFLOW' and 'PERFORMANCE CONSIDERATIONS' sections are arguably beyond the scope of a per-tool description and could be better placed elsewhere. While the description is well structured with headings and bullet points, the length and repetition violate the 'every sentence should earn its place' standard. It is not concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (11 parameters, no output schema, and the need for chaining with other tools), the description is exceptionally complete. It describes the response structure (including testRunUdfs and hasTcRunUdf), defines the exact field names present in each row, and provides guidance on how to handle large data (pagination and batch processing). It also covers error-handling and progressive updates. The output is well-defined even without a formal output schema, and the many examples illustrate behavior under a variety of filter types. The tool's behavior is fully documented.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With full schema coverage (all 11 parameters have descriptions), the baseline is 3, but the description adds substantially more meaning. It clarifies the tsrunID accepts both string and number forms (and warns against alternative names), explains that viewId is auto-resolved unless overridden, and provides a exhaustive reference for the tcrUdfFilter parameter covering all seven field types (list, string, date, numeric, etc.) with required properties, formats, and examples. This goes far beyond the schema's descriptions, which only give a generic JSON string. The output description also conveys the meaning of pagination parameters (start, page, limit) in context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with an explicit and specific purpose: to retrieve test case runs for a given test suite run, optionally including Test Run UDF values. It clearly contrasts with sibling tools (e.g., fetch_executions_by_test_suite, fetch_test_case_executions) by scoping to a specific test suite run ID, and it explains the required parameter (tsrunID) and how to obtain it. The verb and resource are precise, and the text is unambiguous about what data is returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides granular, operational guidance. It includes a 'Use Cases' section enumerating 12 distinct scenarios, and then gives a step-by-step workflow for the common pattern of retrieving all executions of a test suite, mandating a call to FETCH_EXECUTIONS_BY_TESTSUITE first. It also explicitly states when to chain with FETCH_TEST_RUN_UDF_VALUES and how to filter by UDFs, and discourages skipping the discovery of all executions. No alternative tools are mentioned as de facto wrong for this, but the workflow engine is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, covering the safety profile. The description adds meaningful context beyond that, such as the distinction between numeric ID and entityKey, the version default, and the fact that it provides version-specific metadata and history. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections (purpose, parameters, output, use cases, examples, hints). It front-loads the core purpose in the first sentence. While it is somewhat long, each section provides distinct value; nothing is redundant. The formatting aids skimmability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description carries the responsibility of explaining the return. It states 'Output Description: JSON object with version-specific test case details' and gives an expected output in the example. It also covers the ID resolution prerequisite and version fallback. For a read-only fetch tool with 5 parameters, this is reasonably complete, though it omits error conditions and response structure details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents each parameter. The description adds clarifying details: the id parameter's 'Accepts a string or number' (though schema says number) and the distinction between internal numeric ID and entityKey. It also provides an example JSON payload, which reinforces parameter usage. This goes beyond the schema's basic type/description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get QMetry test case details for a specific version by numeric ID'. It distinguishes itself from sibling tools like qmetry_fetch_test_case_details (current version) and qmetry_fetch_test_case_steps by focusing on version-specific details. The mention of 'version details' and 'version history' makes it unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly provides usage context via use cases and hints. It states when to use (get specific version, compare versions, audit changes) and gives practical guidance: 'Requires numeric ID, not entityKey' and 'If user provides entityKey, first resolve it to numeric ID using FETCH_TEST_CASES'. It also notes the default version behavior. This is strong, actionable guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description clearly states that the tool is a fetch (read) operation and explains automatic behaviors like viewId resolution and folderPath defaulting to root. It does not mention error handling or side effects (e.g., what happens if the project key is invalid), but for a read operation this is sufficient. No annotation contradiction was found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is verbose and repetitive. The 'CRITICAL WORKFLOW' hints repeat information already present in the Parameters and Examples sections. While structured and front-loaded, it could be trimmed without losing essential guidance, making it less concise than ideal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity, the description is fairly comprehensive, covering pagination, filtering, sorting, and auto-resolution. However, some parameters like 'scope' and 'getSubEntities' are only partially explained, and the description assumes prior QMetry knowledge. The provided examples cover common scenarios but not edge cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Every parameter is explained in detail, including defaults, automatic behaviors, and when to override. The description adds significant context beyond the schema, such as the automatic viewId resolution and the meaning of 'filter' for entity keys. The examples clarify parameter usage like start/page/limit pagination.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description immediately states the tool's purpose ('Fetch QMetry test suites') and explicitly mentions the automatic viewId resolution. The title and first line are specific and actionable, leaving no ambiguity about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear use cases, examples, and critical workflow hints (e.g., never mix project keys, auto-resolution of viewId). It explains when and how to use the tool, including explicit instructions for manual overrides and common filtering scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not contradict annotations (readOnlyHint:false, etc.). It adds useful behavioral context beyond annotations, such as the requirement for comma-separated IDs without spaces, and the process to resolve IDs via API calls. It does not detail side effects like whether existing links are overwritten, but given annotations, this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear headings (Toolset, Parameters, Output, Use Cases, Examples, Hints) and front-loaded purpose. While it is verbose, with some repetition between Hints and Parameters, each section adds value. The repetition is minor and does not detract from clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
There is no output schema, so the description provides an output overview ('JSON object with linkage status, success message, and details') and examples of expected outputs. It covers all parameters, use cases, and behavioral hints, making the tool fully comprehensible without missing critical information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description significantly enriches parameter understanding. It explains that qmTsId and qmPlatformId are required, provides detailed instructions on how to retrieve them (calling specific APIs and extracting from response), clarifies that qmPlatformId accepts comma-separated values, and gives examples. This goes well beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Link one or more platforms to a QMetry Test Suite.' It specifies the resource (platforms and test suite) and differentiates from sibling tools like qmetry_fetch_platforms (fetch only) and qmetry_link_test_cases_to_test_suite (linking test cases instead of platforms).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit 'Use Cases' and 'Hints' that explain when to use the tool, such as linking single/multiple platforms and defining execution environments. It also gives context on how to obtain required IDs. However, it does not explicitly mention when not to use it or contrast with alternative linking tools, so it lacks exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations declare readOnlyHint=true, idempotentHint=true️, and destructiveHint=false, so the tool is safe and read-only. The description adds that 'scope' and 'orgcode' headers are injected automatically from session context, and notes a failure mode ('auth error' if session not set). Despite no annotation contradiction, it could further clarify side effects or data freshness, but it covers key behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but well-organized with sections for Use Cases, Examples, and Hints flags. The examples and hints add practical value, though some detail (like the repeated 'default: false' for isArchReq) is redundant. Overall, the structure helps an agent navigate the content despite its size.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the cascading UDF workflow, the description covers essential context: prerequisites (calling Fetch Test Run UDF Metadata), the exact value format for bulk updates, the importance of not calling this for other field types, and the auth headers needed. It even handles an edge case (empty lookupOptions) and provides failure guidance. Missing information about empty children lists or pagination is minor compared to the overall completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is high (100%), so baseline is 4. The description adds value by explaining the 'id' parameter's origin (from 'Fetch Test Run UDF Metadata') and the default for 'isArchReq', which goes beyond the schema. It also clarifies that 'id' must be the parent item ID, not a field ID.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the tool's specific purpose: fetching child values of a CASCADINGLIST UDF field given a parent item ID. It distinguishes it from other tools by explicitly limiting usage to CASCADINGLIST fields and mentioning the related 'Fetch Test Run UDF Metadata' and 'Bulk Update' tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance with a mandatory workflow (steps 1-2) and clear exclusions: 'Do NOT call this tool for STRING, NUMBER, DATEPICKER, LOOKUPLIST, or MULTILOOKUPLIST fields.' It also instructs the agent to ask the user for parent ID when lookupOptions is empty, avoiding guesses.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description adds substantial non-obvious behavior: automatic viewId resolution from latestViews.IS.viewId, manual override semantics, no 'DefectId' field (use 'id'), and release/cycle filters requiring IDs rather than names. This goes well beyond what annotations provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded and well-structured with Parameters, Output Description, Use Cases, Examples, and Hints. However, it is bloated with 20 hints and repetitive filler such as 'essential for defect management' and 'critical for quality assurance,' and several examples overlap in content, so not every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by documenting the response shape ('data' array, 'id' as defect identifier). It covers pagination, filtering, sorting, projectKey consistency, and viewId resolution, making it complete enough for correct invocation despite the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3, but the description adds significant meaning: exact filter JSON structures with field/type/value examples, comma-separated entityKeyId search, sort field lists, and the viewId auto-resolution workflow. Examples 5-9 provide concrete payloads that the schema alone does not convey.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Fetch QMetry defects or issues' and clearly indicates it returns a list/data array of issues. This distinguishes it from siblings like qmetry_fetch_issue_details and qmetry_create_defect_or_issue, making the tool's scope specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Use Cases explicitly state when to use the tool: list all issues, search with filters, and get paginated results. Hints also direct users to fetch release/cycle IDs from FETCH_RELEASES_AND_CYCLES before filtering and emphasize consistent projectKey usage. It does not explicitly name alternatives, but the list-vs-detail distinction is clear from the sibling context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnly, idempotent, and non-destructive. The description adds substantial behavioral context by detailing the response structure (viewIds, folderPaths, dateTimeFormatID/New), explaining how to extract viewId from latestViews.TC.viewId, and providing an exhaustive date format mapping that is critical for downstream API calls.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections (Toolset, Parameters, Output, Use Cases, Examples, Hints). It is lengthy, especially the date format hints, but that detail is necessary and each section serves a purpose. Slight redundancy in the numbered hints keeps it from a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description takes on the full burden of explaining return values. It thoroughly covers the response fields, provides two examples with expected outputs, and includes critical date format instructions. It is complete for an agent to know when to call and how to interpret results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and includes a description and default for projectKey. The description adds value with examples, clarification that 'default' is used when unspecified, and context that folderPath for root is an empty string (though that's not a direct parameter). It goes beyond the schema but is not exceptionally rich.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it fetches QMetry project information including viewId and folderPath, and explicitly notes this is needed for other operations. This distinguishes it from sibling tools like qmetry_fetch_qmetry_list_projects and qmetry_set_qmetry_project_info.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit use cases and the hint 'Always call this first when user doesn't provide viewId or folderPath', giving clear when-to-use guidance. However, it doesn't explicitly contrast with alternative tools (e.g., list projects, set project info), so it lacks full when-not-to-use exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description thoroughly discloses behavior beyond annotations: it mandates a unified table output, explains UDF pivoting with forbidden patterns, details the hasTcRunUdf flag, and notes automatic UDF metadata fetching and HTML stripping. It is fully consistent with readOnlyHint and idempotentHint, and no contradiction exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness1/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is excessively verbose and repetitive. It restates the same table format and UDF rules in multiple sections (Examples, Hints, Use Cases, and final reminders). The 'HINTS' section includes numbered lines that are redundant (e.g., points 2–5 and 11–15 repeat the same pivot rule). It lacks conciseness, making it harder to parse essential information quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 params, flexible filter system, intricate output requirements) and the absence of an output schema, the description is exceptionally complete. It covers pagination, filter syntax, UDF enrichment, hasTcRunUdf flag, and integration workflows with other tools. It leaves no critical ambiguity for correct invocation and interpretation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 100% schema coverage, the description adds substantial value: it explains filter fields and types (e.g., testSuiteName, platformID, executionStatus), date comparison operators ('gt'/'lt'), common status values ('PASS', 'FAIL'), archive semantics, and how to resolve platform/release/cycle IDs via other tools. It also clarifies the tcid naming pitfall, going beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states: 'Get execution records for a specific test case by numeric ID, including Test Run UDF values.' It specifies the exact resource (execution records) and action (fetch by tcid). It also distinguishes from siblings by explicitly noting that another tool 'Fetch Test Run UDF Values' is for test suite runs, not test case executions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a list of 13 use cases and explicit 'when NOT to use' guidance, e.g., 'DO NOT call Fetch Test Run UDF Values after this tool for test case executions.' It also gives a critical workflow for linked issues: first fetch executions to extract tcRunID, then use FETCH_LINKED_ISSUES_BY_TESTCASE_RUN. This is explicit and excludes alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. Description adds valuable context beyond annotations: requires numeric ID not entityKey, version defaults to 1, pagination behavior, and output shape (array of steps with description, expected result, order). This is above baseline but not exhaustive; no rate-limit or error behavior noted.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is well-structured with clear sections: summary, parameters, use cases, examples, hints. Every section adds useful information and no filler. While longer than a simple two-sentence description, each part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description adequately explains expected output shapes. It includes use cases, examples, entityKey resolution caveat, pagination hints, and default behaviors. Slight gap: no details on error conditions or how many steps are returned by default, but overall strong completeness for a fetch tool with rich annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds extra semantic value beyond schema: clarifies 'id' means internal numeric identifier and explicitly tells agent to resolve entityKey via FETCH_TEST_CASES, highlights version default, and explains pagination parameters' intended use. These additions make the parameters more actionable, so a 4 is warranted.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb+resource ('Get detailed test case steps for a specific test case by numeric ID') and clearly distinguishes the tool from siblings like fetch_test_case_details or fetch_test_case_version_details. It also clarifies ID type (numeric vs entityKey), reinforcing its unique purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Includes 'Use Cases' section with explicit scenarios (manual run instructions, documentation, automation mapping) and 'Hints' that tell the agent when not to use the tool directly: resolve entityKey via FETCH_TEST_CASES first. This is strong when-to-use and when-not-to-use guidance with an alternative named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Descriptive behavior goes far beyond annotations: output structure details, lookupOptions map semantics, how empty lookupOptions should be handled, and how LOOKUPLIST/MULTILOOKUPLIST values are resolved (uniqueLabel) vs. bulk update IDs. It also documents date format requirements. This is consistent with readOnly/idempotent annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with Toolset, Parameters, Output, Use Cases, Examples, and Hints. The first line gives a concise purpose and the Hints are operational. Some redundancy exists (use cases repeat the purpose and points in Hints), but every section adds useful detail for a complex tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of an output schema, the description thoroughly explains return structure ('fields' array and 'lookupOptions' map), field ID mapping to projectUserFieldID, edge cases (empty lookupOptions, DATE format, authoritative sourcing), and actionable decision guidance, making it complete for an agent to correctly use the tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters with clear descriptions, and the description's parameter section largely mirrors that schema without adding new semantics. There is a minor note about default for projectKey but no additional meaning beyond structured fields.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool fetches metadata (field definitions) for all Test Run UDF fields and specifically notes it returns fieldID/name/label/type needed for bulk updates. It distinguishes itself from related sibling tools like 'fetch_test_run_udf_values' by focusing on metadata, not values, and from 'bulk_update_test_run_udfs' by being the prerequisite data source.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is explicitly prescribed: 'ALWAYS call this tool before Bulk Update Test Run UDFs when the user has not explicitly provided a numeric fieldID' and 'do NOT guess or hard-code field IDs.' It also enumerates concrete use cases (get fieldID for planned_execution_date, list available fields, find lookup list IDs, discover field names).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only/idempotent/non-destructive behavior, and the description adds substantial context: it calls /rest/execution/getExecutionsForIssue, merges Test Run UDF metadata with udfjson, resolves LOOKUPLIST/MULTILOOKUPLIST values, and returns all UDF fields including nulls. It also clarifies hasTcRunUdf and testRunUdfNote behavior. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is heavily repetitive: the unified-table mandate, mandatory column order, and UDF pivoting rules appear in the opening paragraph, Output Description, Examples, and Hints. The Hints section is a numbered list with duplicated admonitions, and many sentences restate the same constraints. It is organized, but not concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema is present, and the description compensates fully: it enumerates every mandatory response field, the shape of testRunUdfs, null handling, filter format, pagination behavior, and auto-resolution. Examples cover typical, filtered, and paginated calls. For a complex 8-param tool, this is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema coverage is 100%, the description goes far beyond the schema: it warns about linkedAssetId naming variants, explains how to resolve issue entity keys, expands every filter field with examples, and notes case-sensitive status names and platform-ID sourcing from FETCH_PLATFORMS. This materially improves invocation correctness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description opens with a specific verb+resource: 'Get test case executions linked to a QMetry-native (non-Jira) defect/issue,' and adds 'Toolset: Issues' to disambiguate. The issue-execution scope and explicit Jira exclusion distinguish it from test-case-centric siblings like qmetry_fetch_test_case_executions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Hints explicitly state when not to use it ('do NOT use for Jira-integrated projects') and direct users to Fetch Defects or Issues for linkedAssetId. It also forbids calling Fetch Test Run UDF Values, saying this tool's response is the direct source for issue-execution UDFs. The listed use cases and filter/pagination guidance provide clear when-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds significant deeper behavioral context: it warns that filtering must use 'isArchived' even though the response shows 'isPlatformArchived', explains that 0 means active and 1 means archived, clarifies that an empty payload {} is sent when no parameters are provided, and instructs about the default sort order. These details go well beyond the annotation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long and well structured, with headers for Toolset, Parameters, Output, Use Cases, Examples, and Hints. It is more verbose than strictly necessary—especially by echoing the schema parameters—but the extra information is organized and useful, so it retains good structure and clear communication.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is no output schema, the description still provides an 'Output Description' stating a JSON object with platforms list and pagination metadata. With 7 optional parameters and no output schema, the examples and hints cover pagination, sorting, filtering, archive-status field naming, and common use cases, making the tool contextually complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the schema has 100% parameter coverage, the description greatly enriches the semantics through concrete examples of how to construct the 'filter' and 'sort' JSON strings, common filter fields like 'name' and 'isArchived', and the meaning of archive status. The schema only says 'Filter criteria as JSON string,' so the descriptions—along with the examples—add essential contextual meaning for an agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Fetch QMetry platforms from the current project,' which clearly states a specific action, resource, and scope. It fully differentiates this tool from the many sibling fetch tools for requirements, test cases, test suites, releases, etc., even without naming alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The 'Use Cases' section lists concrete contexts such as 'Fetch all platforms from the current project', 'List platforms for test environment selection', and 'Get paginated platform results for large projects'. This gives clear guidance on when to use the tool, though it does not explicitly say when not to use it or name alternative sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds meaningful context beyond annotations: this tool does not persist UI filters, requires a numeric internal ID rather than an entity key, and returns metadata/properties rather than steps. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with clear sections: Toolset, Parameters, Output, Use Cases, Examples, and Hints. Some redundancy exists around the CRITICAL tcID naming warning, but the repetition serves to prevent a costly parameter mistake.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by explaining the JSON output includes ID, key, summary, description, and metadata. It also covers use cases, entityKey resolution, and relationships to sibling tools, making it complete for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds substantial meaning beyond the schema, especially for tcID: it warns against incorrect parameter names, distinguishes numeric ID from entityKey, explains how to obtain the ID, and provides resolution options. It also clarifies pagination defaults and the output shape.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Get detailed information for a specific QMetry test case by numeric ID' and explicitly positions it as the tool for single test case lookup. It distinguishes itself from siblings by contrasting with FETCH_TEST_CASES and FETCH_TEST_CASE_STEPS.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit when-to-use guidance: 'USE THIS for single test case lookup', 'ALWAYS prefer this tool over FETCH_TEST_CASES with filters', and directs step-level needs to FETCH_TEST_CASE_STEPS. It also gives concrete options for resolving entityKey to numeric tcID.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, and the description is consistent. It goes beyond annotations by explaining the linked vs. not-linked toggle behavior, pagination semantics, release/cycle filtering, filter field names and ID mappings, and the critical distinction between numeric rqID and entity key.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with sections for Parameters, Use Cases, Examples, and Hints. The parameter list largely duplicates the schema, but the examples and hint list provide actionable guidance, especially for filter JSON construction and ID resolution. Slight redundancy in use cases and hints prevents a 5.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by stating the output shape ('JSON object with test cases array, traceability information, and pagination metadata') and providing 15 examples with expected outcomes. It also covers prerequisite knowledge (release/cycle ID retrieval, entity key resolution) and edge cases like folder paths and archived status, making it fully self-contained for invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, providing a baseline of 3. The description adds substantial extra meaning: rqID is the internal numeric ID, not 'MAC-RQ-730'; releaseID/cycleID should be string IDs; getLinked toggles linked/unlinked results; filter field names and value IDs are enumerated; and multiple filters combine with AND logic. This significantly exceeds schema-only guidance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Get test cases that are linked (or not linked) to a specific requirement in QMetry', using a specific verb and resource with explicit scope. This clearly distinguishes it from sibling tools like qmetry_fetch_requirements_linked_to_test_case (reverse direction) and qmetry_fetch_test_cases (general listing).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a 'Use Cases' section with 10 concrete scenarios such as traceability analysis, gap analysis, coverage verification, and impact analysis. It provides clear context on when to use the tool, though it does not explicitly name alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the tool's safety profile is clear. The description adds substantial behavioral context: automatic viewId resolution from project info, root folder resolution via rootFolders.TS.id, pagination behavior, filter capabilities (fields, AND logic), archive status semantics (0=Active, 1=Archived), and the workflow that handles missing tsFolderID/viewId. This goes well beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is excessively long (over 1500 words) with heavy redundancy. The auto-resolution of viewId is explained multiple times across parameters, hints, and examples. The 'Hints' section repeats the parameter explanations and adds 25 bullet points. While structured with sections, it violates conciseness—many sentences could be merged or removed without loss of information. The core purpose and usage could be conveyed in a few paragraphs.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description provides adequate output context ('JSON object with test suites array and pagination metadata') and many examples with expected outputs. It covers all 9 parameters, elaborate filtering options, pagination, and the auto-resolution workflow. For a complex tool with 9 parameters and heavy integration requirements (project info, folder IDs), the description is complete enough to guide an agent through successful invocation without additional research.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although schema description coverage is 100% (all 9 parameters have descriptions), the description adds critical extra meaning: it warns about the exact parameter name tsFolderID and lists forbidden variants, explains the auto-resolution mechanism for viewId, provides concrete JSON filter examples (release, cycle, isArchived), and details how to obtain tsFolderID from project info. These details are not in the schema and significantly improve correct usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Get test suites that can be linked to test cases in QMetry with automatic viewId resolution.' It specifies the verb (get), the resource (test suites), the scope (for test case linking), and a distinctive feature (auto viewId resolution). This differentiates it from sibling tools like qmetry_fetch_test_suites, which likely fetches test suites without the linking context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: a dedicated 'Use Cases' section lists 9 scenarios, and the 'Hints' section instructs to call FETCH_PROJECT_INFO first, explains when to use auto-resolved vs manual viewId, and states 'Use this before creating new test suites to check existing ones.' It also implicitly contrasts with other tools by focusing on fetching test suites for linking, not creating or modifying them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description goes well beyond the annotations by warning about silent data loss from incorrect date formats, the requirement to apply default values to avoid data loss, the API returning success even when values are not stored, and the need to include both udfFields and UDF during update. This is highly valuable behavioral context for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very long and duplicates some parameter/schema content, which slightly hurts conciseness. However, it is extremely well structured with separate Parameters, Use Cases, Examples, and Hints sections, numbered instructions, and examples that make the length easier to navigate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a complex 11-parameter mutation tool with nested UDF objects and no output schema, the description is comprehensive. It covers mandatory-field fallbacks, default-value handling, date format derivation, UDF workflow, required companion calls, and even the expected output format. This is sufficient context for successful invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description still adds meaningful parameter context: the exact DefectId casing requirement, UDF field type mappings, default routing for system fields versus UDF fields, date format rules, and the required UDF wrapper structure. It far exceeds what the schema alone provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific operation: 'Update an existing QMetry issue by DefectId and/or entityKey.' It clearly identifies the resource and the required identifiers, and it is easily distinguishable from sibling tools like qmetry_create_defect_or_issue or qmetry_fetch_issue_details.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides use cases, prerequisites, and references to supporting tools such as 'Fetch UDF Layout', 'Fetch QMetry Project Info', 'Issue/Fetch issue', and 'Create Issue tool'. It gives clear context for when to use the tool, though it does not explicitly state when not to use it or contrast with the create tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds substantial behavioral context beyond the annotations. It explains that releaseID is required and how to obtain it, warns about date format variability ('Date format depends on QMetry instance configuration: DD-MM-YYYY or MM-DD-YYYY'), and clarifies that projectID is optional and auto-resolved. It also mentions defaults for isLocked and isArchived (which are not in the schema but are disclosed). There is no contradiction with annotations (readOnlyHint=false is consistent with a create operation).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally long, spanning nearly 40 lines, with repetitive guidance on fetching release IDs (the workflow is described multiple times). While it is well-structured with headings (Parameters, Output Description, Use Cases, Examples, Hints) and front-loaded with a clear purpose sentence, it includes many redundant hints that could be condensed to improve focus.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity of creating a cycle within a release, the description is thorough. It covers use cases, examples, expected outputs, error handling (date format), prerequisites (releaseID), and the relationship to other tools. It even provides post-creation verification advice. With no output schema, the 'Expected Output' examples fill that gap effectively. The description leaves little to ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description significantly enriches parameter understanding. While the schema has descriptions for baseUrl, projectKey, and inner cycle fields, the description adds meaning by explaining the cycle object structure, required vs optional fields, how to resolve releaseID, and provides four concrete examples with expected outputs. It also clarifies that projectID can be auto-resolved and explains date format caveats. This goes well beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create a new cycle within an existing release in QMetry for test execution planning.' It uses a specific verb (create) and resource (cycle), and explicitly differentiates from the sibling CREATE_RELEASE tool by explaining the difference: 'This tool creates a cycle in an EXISTING release, while CREATE_RELEASE can create a release with an optional cycle.' This satisfies the requirement of distinguishing from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance including when to use this tool vs. alternatives. It states 'DIFFERENCE FROM CREATE_RELEASE' and advises 'If you need to create both a release and a cycle together, use CREATE_RELEASE tool instead.' It also includes a detailed workflow for resolving release names to IDs via FETCH_RELEASES_CYCLES, covering when the user provides a release name, and includes steps to follow if the release is not found. This is comprehensive and clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description comprehensively discloses behavioral traits beyond annotations: it mandates a pre-create call to 'Fetch UDF Layout', warns about silent data loss with date formats, specifies the post-create platform linking requirement, and details error recovery for 'CO.MANDATORY_FIELDS_MISSING'. Annotations are minimal (readOnlyHint=false, destructiveHint=false), so the description carries the full burden and does it exceptionally well, with no contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely long (over 100 lines) and includes repetitive instructions (e.g., repeated calls to 'Fetch UDF Layout', multiple mentions of the same ID mapping steps). It uses ASCII art boxes and numbered steps that add structure but become verbose. While every sentence adds some value, the density hinders quick scanning; it could be condensed by removing redundant warnings and consolidating repeated steps.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is richly complete for the tool's complexity: it covers prerequisites (pre-create UDF layout call), mandatory and default field handling, error recovery procedures, date formatting, post-create platform linking, and examples with expected outputs. It accounts for the 9 parameters, nested objects, and the absence of an output schema by describing the output in the output description section. It's thoroughly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is only 33%, but the description provides extensive parameter semantics beyond the schema: for parentFolderId it explains auto-resolution, for releaseCycleMapping it clarifies the exact key names (releaseId vs release, buildID vs cycle) and the critical shape difference from Test Cases, and for udfFields it elaborates on types and source of values. It also details how to obtain IDs for testsuiteOwner and testSuiteState, filling in all gaps left by the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new test suite in QMetry with metadata and release/cycle mapping, using a specific verb ('Create') and resource ('Test Suite'). It distinguishes from siblings like qmetry_update_test_suite and qmetry_create_test_case, and provides use cases and examples that reinforce the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description is explicit about when to use this tool (for creating test suites) and when not to (e.g., do not use Test Case mapping shape), and provides clear alternatives or related steps like calling 'Fetch UDF Layout' before creation and 'Link Platforms to Test Suite' after. It also includes step-by-step prerequisites and decision matrices, making it highly actionable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the read-only nature is established. The description adds meaningful behavior: getLinked default semantics, tcID vs entityKey distinction, root-folder default behavior, filter field capabilities, AND combination logic, pagination support, and output shape. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with headings for Parameters, Output Description, Use Cases, Examples, and Hints, and it front-loads the core purpose. However, it is quite long: ten use cases and fifteen hints contain notable redundancy, so it could be tightened without losing value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity — 9 parameters, rich filtering, pagination, and no output schema — the description is thorough enough. It explains the output as a JSON object with requirements, traceability, and pagination metadata, provides examples across many filter scenarios, and covers the critical tcID resolution workflow.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the schema itself is already detailed, so the baseline is 3. The description adds value beyond the schema with filter field examples, expected behavior for entityKey searches, folder path guidance, and ten worked examples showing real JSON payloads and expected outputs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb-plus-resource statement: 'Get requirements that are linked (or not linked) to a specific test case in QMetry.' It clearly distinguishes this from sibling tools such as qmetry_fetch_test_cases_linked_to_requirement and qmetry_link_requirements_to_testcase by framing it as a fetch/query operation on test-case-to-requirement relationships.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit use cases, including traceability analysis, gap analysis, and compliance audits. It also gives actionable workflow guidance: if a user supplies an entity key like MAC-TC-1684, the agent should first call FETCH_TEST_CASES to resolve the numeric tcID before calling this tool. This is clear when-to-use and workflow context beyond the schema.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (readOnlyHint=true, destructiveHint=false), the description discloses significant behavioral traits: the auto-resolution of viewId, the filter persistence side effect in the production UI, and the deprecation of entityKeyId filter for single test case. These are critical non-obvious behaviors that annotations alone don't convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with clear headings (Parameters, Use Cases, Examples, Hints). It is front-loaded with the core purpose and then expands into necessary details. While verbose, the complexity of the tool (17 params, multiple filter variations) justifies the length. Some redundancy exists (e.g., repeated filter warnings), but it remains organized.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (17 params, no output schema), the description is remarkably complete. It covers output format ('JSON object with data array and pagination info'), provides 9 examples covering common scenarios, explains filter syntax for release/cycle/entityKey, and gives sort fields. It also includes critical workflow steps and cross-tool references (e.g., fetching release/cycle IDs).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the schema descriptions already explain each parameter. The description adds extra meaning for key parameters like viewId (auto-resolution override) and folderPath (root default), and provides filter format examples. It enhances understanding beyond the schema but doesn't radically extend it for all 17 parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Fetch QMetry test cases' with a specific verb and resource. It distinguishes from siblings like qmetry_fetch_test_case_details (single test case) and qmetry_fetch_test_suites by explicitly scoping to listing/bulk retrieval. The purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes an explicit 'WHEN TO USE THIS TOOL' section listing exact user intents, and warns against using it for single test case lookup, directing to 'Fetch Test Case Details' instead. It also provides critical filter persistence warnings and workflow steps, giving clear when/when-not and alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the operation as readOnly, idempotent, and non-destructive, but the description adds substantial behavioral context: fieldID availability differs between ADD and DETAIL, listOptions may be fallback-populated, defaultValues auto-fill without user input, and DATETIMEPICKER values must match project date format. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long, but it is well-organized into clear sections (Use Cases, Examples, Hints, Output Description) and front-loaded with the core purpose. Some redundancy exists between Use Cases and Hints, and the numbered workflow indentation is awkward, but the length is largely justified given the tool's complexity and absence of an output schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description carries full responsibility for explaining return structure, and it does so thoroughly: fields, systemFields, defaultValues, stepFields, listOptions, and entity-specific keys are all documented. It also covers mandatory checks, fallback behavior, date formatting, and full create/update integration workflows, making it complete for safe autonomous use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already documented structurally. The description adds value by explaining how entityType affects the output shape (TC includes stepFields), how pageName controls fieldID presence, and how projectKey scopes all returned data. The examples also map parameter combinations to realistic output structures.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description opens with a specific verb and resource: 'Fetch UDF definitions for Test Case, Test Suite, or Issue entities' and lists concrete returned data (field names, types, fieldIDs, lookup option IDs). It also clearly positions the tool as a prerequisite step before creating or updating entities, distinguishing it from other fetch/metadata sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides extensive usage guidance: when to use ADD vs DETAIL, when to call before create vs update, explicit workflow steps, and even an alternative tool (Fetch Test Run UDF Metadata) if listOptions come back empty. It also covers mandatory-field resolution and cascade/date-format edge cases, making usage conditions highly explicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false, destructiveHint=false, but the description goes far beyond by disclosing critical behaviors: silent data loss on wrong date formats, mandatory default value sweeping, auto-resolution of TsFolderID, and the need to call supporting tools. It also warns about mandatory UDF fields and the consequences of omitting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely long and repetitive, with many numbered hints that repeat the same information (e.g., auto-resolution of TsFolderID mentioned multiple times). While it is well-structured with sections, the verbosity and redundancy reduce its conciseness. It could be trimmed significantly without losing value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (9 parameters, nested objects, no output schema), the description is exceptionally complete. It covers all necessary preconditions, workflows, error handling, and fallback messages. The examples and hints provide comprehensive guidance for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds substantial meaning: it explains the auto-resolution of TsFolderID, the dual requirement of udfFields and UDF wrapper, the exact structure for CASCADINGLIST, and the need to fetch UDF layout for field IDs. It also clarifies the date format handling and the mapping of owner/state names to IDs.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing QMetry test suite by numeric id with auto-resolution from entityKey. It distinguishes from siblings like qmetry_create_test_suite and qmetry_fetch_test_suites by specifying the update action and the auto-resolution mechanism.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides extensive usage guidance: when to use (update operations), when not to (e.g., creation via qmetry_create_test_suite), and explicit alternatives like 'Fetch UDF Layout' and 'Fetch QMetry Project Info'. It also details prerequisites and fallback behaviors for owner/state resolution.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations only declare readOnlyHint=false, destructiveHint=false, and idempotentHint=false; the description adds crucial behavioral context beyond these: the operation runs asynchronously (success means queued, not completed), uses 'Scheduled Task' for tracking, and warns of silent failure if fieldIDs are incorrect. These operational caveats are not inferable from annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long (use cases, 8 examples, 11 hints) but well-structured with clear sections (Parameters, Output, Use Cases, Examples, Hints). It front-loads the core purpose and parameters, and every section contributes to usage correctness. Some redundancy exists (examples and hints repeat similar scenarios), but overall it is appropriately detailed for a complex tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (nested objects, multiple field types, asynchronous behavior, dependency on other tools), the description leaves virtually nothing unexplained: it covers prerequisites, parameter formats, error-prone aspects (fieldID correctness), the need to fetch tcRunIDs via a separate tool, and output meaning. It also explicitly differentiates from a sibling tool using entityIDs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Even though schema coverage is 100%, the description adds significant semantic value: detailed value formats for each UDF field type (STRING, NUMBER, DATETIMEPICKER with MM-DD-YYYY, LOOKUPLIST, MULTILOOKUPLIST, CASCADINGLIST), rules for multiSelectAction (default to append, never assume replace), and step-by-step guidance on obtaining valid fieldIDs and cascade child IDs. Multiple JSON examples illustrate exact structures for each type.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (bulk update), the resource (User Defined Field values for Test Case Runs), and the scope (one or more test runs). It distinguishes itself from sibling tools like 'Bulk Update Test Case Execution Status' by explicitly noting the use of tcRunIDs vs entityIDs.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides extensive when-to-use guidance: 10 concrete use cases (e.g., bulk updating string, date, number, lookup, multi-select fields), explicit workflow instructions (call 'Fetch Test Case Runs by Test Suite Run' first to collect tcRunIDs), and exclusions (do not confuse with entityIDs, do not hard-code tcRunIDs). It also gives decision rules for append vs replace based on user intent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive. The description adds behavioral context: it forwards to backend, returns response exactly as received without transformation, and explains pagination behavior. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured: purpose, parameters, use cases, examples, and hints. It repeats parameter info from schema, but the layout is efficient for the tool's complexity. Examples and expected output justify the length; still some redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description provides expected output format in examples. It covers pagination, scoping options, and mentions the need to fetch configuration first. Complete for a complex tool with 9 parameters and no structured return schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3, but the description massively enriches parameter meaning with real examples (e.g., projectId 45851, releaseId 90698) and explains how optional parameters like cycleIds scope reports. The 'Hints' section clarifies required vs optional usage beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool executes a quality gate report by forwarding to the backend analytics engine. It distinguishes from siblings like qmetry_fetch_quality_gate_configuration by focusing on execution. Specific verb 'Execute' and resource 'quality gate report' make purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit use cases (release readiness, gate assessment, cycle scoping) and a hint to 'Call Fetch Quality Gate Configuration first' to discover available gates. This gives clear when-to-use guidance and points to an alternative tool for pre-configuration.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark this read-only and idempotent, but the description adds valuable behavior detail: it calls UDF metadata internally, optionally refetches the parent execution list when sourceRows is omitted, reuses rows when provided, resolves lookup display names, and warns about a _lookupWarning response. This goes well beyond the annotation hints and fully discloses the tool's internal behavior and edge-case handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured into Parameters, Output Description, Use Cases, Examples, and Hints, making it scannable. However, several warnings and workflow instructions are repeated across multiple sections (e.g., 'do NOT use for test case executions'), which adds length without adding new information. The size is largely justified by tool complexity, but trimming redundancies would make it tighter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 8 parameters, no output schema, and no nested simple structure, this description provides an unusually complete operational picture: it defines the unified table output contract, lists mandatory columns, explains when hasTcRunUdf false is meaningful, and even covers lookup resolution failure handling. The use-cases and examples give the agent concrete invocation templates, making it fully self-sufficient for correct use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the input schema already provides 100% coverage, the description adds critical semantic depth beyond the schema: it emphasizes the exact parameter name 'tsrunID' with commonly mistaken variants, explains how to obtain the value from 'Fetch Executions by Test Suite' (data[<index>].tsRunID), and describes the sourceRows/sourceContext relationship in terms of workflow state. These additions materially improve the agent's ability to call the tool correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific and unambiguous statement: 'Fetch the Test Run UDF values for all test case runs in a given test suite run.' It names the exact resource and scope, and distinguishes the tool from related siblings by explicitly directing users away from Fetch Test Case Executions and Fetch Issue Executions for UDF data.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides both positive guidance ('Use this tool for test suite run UDF values') and explicit exclusions ('Do NOT use this tool for test case executions' and 'Do NOT use this tool for Fetch Issue Executions'). It also details the parent-to-UDF workflow with concrete prompt patterns and alternative tools, so the agent knows exactly when to invoke this tool versus when to use the sibling tool directly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (readOnlyHint=false, idempotentHint=true), the description discloses key behaviors: update is partial ('Only include the fields you want to update - other fields will remain unchanged'), identification requires buildID/releaseID, validation of date format depends on instance configuration, and verification via FETCH_RELEASES_CYCLES after update. It also explains the hierarchy (Project → Release → Cycle → Test Execution) and the difference from creation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally structured with clear sections (Parameters, Output Description, Use Cases, Examples, Hints) and uses enumeration. However, it is very long and contains redundancy—e.g., the workflow for fetching IDs is repeated multiple times in different formats (text and numbered steps). Despite being verbose, the organization is usable, so it earns a 4 rather than lower.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutating tool with nested parameters and no output schema, the description is remarkably complete. It covers use cases, multiple worked examples, expected outputs for each example, error conditions (date format mismatch), a detailed workflow for a common user request (by name), and post-update verification steps. It also clarifies differences from similar tools and provides operational hints that an agent would need.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description enriches parameter semantics well beyond the schema. It provides a full example JSON for each parameter combination, explains how to obtain buildID/releaseID from FETCH_RELEASES_CYCLES, clarifies date format ambiguity, and notes optionality of fields. It even gives hints on how to match cycle names and handle multiple matches. Schema coverage was 67%, but the description compensates fully for the uncovered baseUrl/projectKey by explaining their role.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Update an existing cycle in QMetry for test execution planning' with a specific verb (Update) and resource (cycle). It explicitly differentiates from sibling CREATE_CYCLE: 'DIFFERENCE FROM CREATE_CYCLE: This tool updates an EXISTING cycle, while CREATE_CYCLE creates a new one'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides extensive usage guidance: it explains when to use this tool (to update existing cycles), and importantly, contrasts with create_cycle. It also gives critical workflow instructions for when the user provides a cycle name—requiring a prior FETCH_RELEASES_CYCLES call to obtain buildID/releaseID. It includes explicit do-nots ('NEVER assume or guess buildID or releaseID') and edge-case handling (multiple matches, case-insensitivity).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![smartbear-mcp MCP server](https://glama.ai/mcp/servers/SmartBear/smartbear-mcp/badges/card.svg)](https://glama.ai/mcp/servers/SmartBear/smartbear-mcp)

Score Badge

Copy to your README.md:

[![smartbear-mcp MCP server](https://glama.ai/mcp/servers/SmartBear/smartbear-mcp/badges/score.svg)](https://glama.ai/mcp/servers/SmartBear/smartbear-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/SmartBear/smartbear-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server