TestOps MCP Server

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v1.7.0
Disambiguation5/5
Every tool has a clearly distinct purpose, targeting specific entities or operations (e.g., get-test-case vs get-test-case-overview, or individual set-* for different attributes). Descriptions provide sufficient context to avoid confusion.
Naming Consistency5/5
All tool names follow a consistent verb_noun pattern with underscores (e.g., list-test-results, get-test-case-scenario, set-test-case-custom-fields). No mixing of conventions.
Tool Count4/5
48 tools is high, but each covers a specific aspect of test management (projects, test cases, plans, launches, defects, settings). The granularity is justified by the domain's complexity, though slightly over the typical sweet spot.
Completeness5/5
The tool set provides comprehensive CRUD and lifecycle coverage for all major entities (test cases, plans, launches, defects) plus extras like scenario management, custom fields, relations, trends, and AQL search. No obvious gaps.
Average 3.4/5 across 48 of 48 tools scored. Lowest: 2.2/5.
See the Tool Scores section below for per-tool breakdowns.
- No community issues in the last 6 months
- 1 commit in the last 12 weeks
- No stable releases found
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under MIT License.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not disclose behavior beyond the generic 'get test results'. With no annotations, it should explain pagination (page/size), read-only nature, authorization requirements, or response format. None are mentioned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is overly terse (one short sentence). While concise, it omits essential details that an agent needs to invoke the tool correctly, making it under-specified rather than efficiently descriptive.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema and low schema coverage, the description is severely incomplete. It fails to explain the response format, pagination behavior, or relationship to sibling tools, leaving critical gaps for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is low (33%): only the required 'id' has a description ('Launch ID'), while 'page' and 'size' are undocumented. The description does not elaborate on any parameter meanings, defaults, or constraints, failing to compensate for the schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the tool gets test results for a specific launch, but it lacks specificity: it does not clarify whether results are returned individually or as a list, and it does not distinguish from sibling tools like get-test-result or list-test-results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as get-test-result (single test) or list-test-results (list all). There are no usage conditions or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does not disclose any behavioral traits such as pagination, result limits, default sorting, or return format. The parameters page and size hint at pagination, but the description does not confirm or explain. This is a critical gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness2/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence), which is concise. However, it omits essential information, making it under-specified rather than efficiently informative. Appropriate sizing would require more detail to be useful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness1/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the three parameters, no output schema, and no annotations, the description is highly incomplete. It fails to explain pagination, return type, required permissions, or how to handle errors. The sibling tools are numerous, and this description does not help the agent select correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 33% (only projectId described). The description adds no meaning to page or size parameters (e.g., pagination, defaults). It repeats 'project' context but does not compensate for the missing parameter descriptions. The description adds minimal value beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List defects in a project' clearly states the verb and resource. The name 'list-defects' distinguishes it from sibling tools like get-defect (single) and create-defect. However, it does not explicitly differentiate from other list tools but achieves clarity through the resource name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., get-defect, search-test-cases). There is no mention of prerequisites, filters, or when not to use it. The agent is left to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavior but only states basic purpose. It does not mention pagination (though 'page' and 'size' params exist), result ordering, auth requirements, or whether it lists all launches or filtered ones.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one short sentence, which is concise but too sparse to be helpful. It front-loads the purpose but lacks necessary detail. Every sentence should earn its place; here, the sentence is minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without output schema, return values are unexplained. The description does not cover pagination behavior, required fields beyond projectId, or any additional context needed for correct usage. The tool is underspecified for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters1/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is low (33%) and the tool description adds no meaning to the parameters. 'page' and 'size' are unexplained, and 'projectId' is only described in the schema. The description should clarify these parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List') and resource ('test launches') with scope ('in a project'). It distinguishes from siblings like 'get-launch' (singular) and 'list-test-results' (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Does not mention prerequisites, when not to use, or comparative context with siblings like 'get-launch'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description does not disclose behavior beyond listing: missing details on pagination (page/size params), read-only nature, or any side effects. The 'Optionally filter' line is inaccurate given schema requirement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise at one sentence, but lacks structure. No separation for when-to-use, output, or parameter behavior. Could be more informative without adding much length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and 4 params (2 required) with missing descriptions for page/size, the description is insufficient. Does not mention pagination behavior, result ordering, or what constitutes a 'test result'. Incomplete for effective tool invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 50% (projectId, launchId have descriptions). Description adds no parameter-level detail; it repeats 'filter by launch' inaccurately and ignores page/size entirely.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose3/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states verb 'List' and resource 'test results in a project', which is clear. However, it claims 'Optionally filter by launch' while input schema requires launchId, creating a contradiction that undermines clarity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get-launch-test-results or other list tools. No context about prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden for behavioral traits. It fails to disclose that the tool is paginated (via page and size parameters), what the response contains (e.g., test case IDs or full objects), or any side effects (none expected). The minimal description leaves agent uncertain about behavior beyond a basic read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise—one sentence. However, it lacks essential information (pagination details, return format, usage context), suggesting under-specification rather than efficient conciseness. It is front-loaded with purpose but at the expense of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 3 parameters, no output schema, and no annotations, the description is insufficient. It does not explain return values, pagination behavior, or what constitutes 'test cases included in a test plan' (direct vs. inherited associations). An agent would likely need additional context to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is only 33% (only the 'id' parameter has a description). The description does not explain the 'page' and 'size' parameters (likely pagination) or clarify the format of the test plan ID. The description adds no meaning beyond the schema's minimal 'Test plan ID' for the required parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves test cases linked to a test plan, using specific verb 'Get' and resource 'test cases in a test plan'. It is distinguishable from sibling tools like list-test-cases (which likely lists all cases in a project) and get-test-case (a single case). However, it does not specify the scope (e.g., all cases or a subset) or the output format (IDs vs. full details), leaving some ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like list-test-cases (project-level) or search-test-cases. It does not mention prerequisites (e.g., must have a test plan ID) or exclude scenarios. The input schema implies it requires a test plan ID, but this is not explicitly stated in the description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must disclose behavioral traits. It only states 'Update an existing test plan' without specifying whether it performs a partial or full update, what side effects occur, or any required permissions. This is insufficient for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence). While it wastes no words, it is under-sized for the tool's complexity and lacks necessary detail, making it less effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, no output schema, no annotations, and many sibling tools, the description is incomplete. It fails to explain the update behavior, return values, or any constraints, leaving the agent with significant ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has only 25% description coverage (only 'id' has a description). The description adds no additional meaning to the parameters; it does not explain that name, status, and description are updatable fields or their expected formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly identifies the action ('Update') and the resource ('an existing test plan'), making the basic purpose understandable. It distinguishes itself from sibling tools like create-test-plan and delete-test-plan. However, it could be more specific about which fields can be updated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like create-test-plan or get-test-plan. There is no mention of prerequisites, typical use cases, or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full responsibility for behavioral disclosure. The description fails to mention any behavioral traits such as what happens if the ID does not exist (e.g., error or null), any required permissions, or whether the tool is read-only. This is a significant gap for a single-parameter tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence, with no unnecessary words. It is front-loaded with the core action and resource. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple interface (one parameter, no output schema, no annotations), the description is insufficiently complete. It does not explain what 'project details' entails, the structure of the response, or any error handling. An agent using this tool would lack crucial execution context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already fully describes the 'id' parameter as 'Project ID' (100% coverage). The description adds no additional meaning beyond what the schema provides, so the baseline score of 3 is appropriate. No extra syntax, format, or constraints are specified.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('project details') with the qualifier 'by ID', making the purpose straightforward. It distinguishes from sibling tools like 'list-projects' which lists all projects. However, it is very brief and lacks any additional context that could further clarify the exact nature of the details returned.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives. For example, it does not mention that 'list-projects' should be used to retrieve all projects, nor does it provide any prerequisites or context about the project ID. The agent is left to infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of disclosing behavior. It only says 'Get' (implying read-only) but provides no details about the response format, pagination, or any side effects. The brevity leaves significant ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, which is concise but sacrifices important details. It earns its place, but the lack of structure (e.g., no bullet points or paragraphs) limits usefulness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and annotations, the description is insufficiently complete. It does not explain what 'trend' entails (e.g., time units, metrics), leaving the agent underinformed about the return value.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a basic description for projectId. The tool description does not add extra meaning beyond what the schema already provides (e.g., no hints about allowable values or range). Baseline of 3 applies because schema adequately documents the single parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves 'test success rate trend' for a project, indicating a specific verb and resource. However, it does not differentiate from similar sibling tools like 'get-automation-trend' or 'get-status-distribution', which could cause confusion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor are any prerequisites or exclusions mentioned. The description lacks context about appropriate use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It implies a read operation ('Get') but does not disclose any behavioral traits such as required permissions, whether it's read-only, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence. However, it could provide slightly more detail without becoming verbose, such as indicating the output type.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has no output schema, and the description fails to describe the structure or content of the returned chart data. For a simple tool, this lack of output context leaves the agent uncertain about the return value.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (one parameter projectId described as 'Project ID (required)'). The description adds no extra meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'automation trend chart data' for a project, but lacks specificity about what the chart data contains, which would help distinguish it from other analytics tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like get-success-rate or get-status-distribution. No prerequisites or context are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not disclose any behavioral traits. Since the tool is a get operation by ID, it is likely read-only, but this is not stated. No annotation or additional context is given about authorization, rate limits, or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise (one sentence) and front-loaded with the action and resource. However, it is borderline under-specified; a bit more context on returned details would improve utility without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple input (1 required parameter) and no output schema, the description is minimally adequate. However, 'details' is vague and does not hint at the return structure. For a tool that retrieves a specific entity, more detail on what fields are returned would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds minimal value beyond the schema. The schema already documents 'id' as 'Launch ID'; the description simply states 'by ID'. No additional parameter semantics are provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('launch details'), making the purpose evident. However, it does not differentiate from sibling tools like get-launch-progress or get-launch-test-results, which also retrieve specific aspects of a launch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not specify when to use this tool over alternatives (e.g., list-launches for all launches, get-launch-progress for progress). The agent is left without guidance on context or selection criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose all behavioral traits. It states a read operation but does not describe the output format, pagination, or any side effects. For a distribution tool, the structure of the result is critical and missing.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no extraneous words. It efficiently conveys the core purpose, though it may be too brief to cover necessary details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema, the description should explain what the distribution contains (e.g., status categories and counts). Without this, the agent cannot interpret the tool's output, making it incomplete for a summary tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides 100% coverage with a description for the single parameter 'projectId'. The tool description adds no additional meaning beyond what the schema already states, so it meets the baseline without improvement.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves a status distribution for a project. The verb 'get' and resource 'status distribution' are specific enough to differentiate from siblings like 'get-success-rate' or 'list-test-cases', though the exact nature of the distribution (e.g., counts per status) is not elaborated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like aggregating from 'list-test-cases'. There is no mention of prerequisites or exclusions, leaving the agent without context for appropriate invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states it is a read operation ('get'), but with no annotations provided, it fails to disclose behaviors such as whether the operation is safe, what permissions are needed, or what exactly 'details' encompasses. No output schema compounds the ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise at one sentence, with no wasted words. While it is efficient, it could be slightly expanded to include result expectations without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With one parameter, no output schema, and no annotations, the description is incomplete. It fails to describe what the returned 'details' include, leaving the agent uncertain about the result structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% since the single parameter 'id' is documented. The description adds no additional meaning beyond the schema, meeting the baseline expectation but not enhancing it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get test plan details by ID' clearly indicates the action (get) and the resource (test plan details) with a required identifier. However, it does not differentiate from sibling tools like 'list-test-plans' or 'get-test-plan-test-cases', though the purpose is straightforward for a single-entity retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, if the user needs a list of test plans, they should use 'list-test-plans' instead. The description lacks explicit usage context or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description fails to disclose side effects, permissions needed, error behavior, or whether partial updates are supported. Bare minimum for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise but lacks structure to include essential details. Would benefit from additional sentences on usage and behavior.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 4 parameters with minimal schema coverage and no annotations or output schema, the description is insufficient for an agent to confidently use the tool. Missing details on return value, error cases, and field updates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is low (25%: only 'id' described). Description does not elaborate on parameter meanings, constraints, or update behavior beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description 'Update an existing defect' clearly states verb (update) and resource (defect), distinguishing it from sibling tools like create-defect, get-defect, and list-defects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives or when not to use it. Lacks prerequisites or context for invocation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only says 'create.' It does not disclose important behavioral traits such as authentication requirements, side effects, error handling, or what the tool returns. This leaves significant gaps for an agent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It could benefit from more structure, but it is concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and annotations, the description is incomplete. It does not specify return values, required permissions, or constraints. For a creation tool with three parameters, more context is needed for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides descriptions for all three parameters (100% coverage). The description adds no additional meaning beyond the schema, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new defect in a project,' which is a specific verb and resource. It distinguishes from siblings like 'list-defects' and 'update-defect' that perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, nor any context about prerequisites or exclusions. For example, it does not mention that the project must exist or that creation may be restricted by permissions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It implies a read-only operation but fails to disclose error behavior, response format, or what happens when the test case or custom fields are absent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence with no filler, front-loaded with the core action and examples. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate for a simple getter but lacks output details (e.g., response structure, key names, potential empty results). No output schema to compensate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. The description adds no extra parameter meaning beyond the schema, earning the baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves custom field values for a test case and provides concrete examples (Component, Priority, Team, Section). It effectively distinguishes it from sibling tools like set-test-case-custom-fields or list-custom-field-values, though it does not explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as get-test-case or list-custom-field-values. The description does not mention prerequisites, context, or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; the description merely implies a read operation via 'Get' but does not explicitly state side-effects, idempotency, or any behavioral constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, clear sentence with no waste. Efficient and to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Minimal description for a simple tool; lacks explanation of what test keys are or the return format, but no output schema exists to offset this.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (single parameter 'id' described as 'Test case ID'), so the description adds no extra meaning; baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('test keys for a test case'), distinguishing it from sibling tools like 'get-test-case' and 'set-test-case-test-keys'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, nor any prerequisites or context hints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It only states 'Create a new test plan' without disclosing side effects, required permissions, or behavior on success/failure. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no fluff, directly stating the purpose. It is optimally concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and low complexity, the description still lacks context such as what a test plan is or that it requires a projectId. It is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already describes all parameters. The description adds no extra meaning beyond the schema, achieving the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('a new test plan in a project'), effectively distinguishing it from sibling tools like list-test-plans, get-test-plan, and update-test-plan.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor any prerequisites or context. It is a bare statement with no usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description must fully disclose behavior. It only states a read operation but does not mention error handling, missing IDs, rate limits, or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence with no wasted words. It efficiently conveys the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter retrieval tool without an output schema, the description is adequate. It lacks mention of return format or error conditions, but the simplicity of the operation compensates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter 'id' described as 'Defect ID'. The description adds no extra meaning beyond the schema, meeting the baseline for high coverage but with no added value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'defect details' with the key parameter 'by ID'. It is distinct from siblings like 'list-defects' and 'create-defect'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., list-defects for browsing). There are no mentions of prerequisites, when-not-to-use, or alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits like required permissions, idempotency, side effects, or error handling. For a mutation tool, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single clear sentence with examples, no wasted words. It is appropriately front-loaded. However, it could benefit from a slightly more structured format.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with three parameters, the description covers the basic usage. However, lacking output schema and behavioral details, it feels incomplete. Adequate but with room for improvement.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All three parameters are documented in the schema (100% coverage). The description adds examples ('change status or add comment') but does not provide further semantics beyond the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and the resource ('test result'), with concrete examples ('change status or add comment'). This effectively distinguishes it from sibling tools like update-test-case or get-test-result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives, such as get-test-result for viewing or update-test-case for modifying test cases. The description simply states the action without context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden but only states it retrieves members. It does not disclose read-only nature, authentication needs, or behavior for invalid IDs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded with the main action and resource.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple retrieval tool with one parameter and no output schema, the description is barely adequate. It lacks details on return format or scope, but given simplicity, it meets minimum completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the 'id' parameter described. The description adds context about member types but doesn't enhance parameter meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (get) and resource (members for a test case), with examples of member types (owner, reviewers). It distinguishes itself from siblings like set-test-case-members and other get tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The sibling set-test-case-members implies a counterpart, but the description does not contextualize usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as authentication needs, side effects, or whether the result is a list. As the sole source of behavioral info, it is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that is front-loaded and contains no unnecessary words. Every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one required param, no output schema), but the description does not mention return format or any result details. It is minimally adequate but leaves gaps for an agent understanding the response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents the single parameter 'id' as 'Test case ID'. The description adds no additional meaning, meeting the baseline but not exceeding it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get requirements linked to a test case' clearly identifies the verb (get) and resource (requirements linked to a test case), distinguishing it from siblings like 'get-test-case' or 'set-test-case-requirements'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'set-test-case-requirements' or 'get-test-case-issues'. The description lacks context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description fails to disclose that the tool is paginated (via 'page' and 'size' parameters) or that it lists 'all' projects may be constrained by page size. This is a notable omission.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise at two sentences and front-loads the purpose. No wasted words, but structure is simple.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only list tool with no output schema, the description should clarify the response format (e.g., array of project objects). It only says 'returns project IDs,' leaving ambiguity. Also lacks mention of default pagination values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (both 'page' and 'size' have descriptions). The tool description adds no parameter information, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and resource 'projects', explicitly mentioning it returns project IDs needed for other operations, which distinguishes it from other list tools like list-test-results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for obtaining project IDs as a prerequisite for other operations, but does not explicitly state when not to use it or mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It states a read operation but does not disclose pagination behavior, default sorting, rate limits, or what happens when no search is provided. The schema indicates pagination but description ignores this.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two front-loaded sentences with no redundancy. Every sentence adds value: purpose, prerequisite, and capability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers basic purpose and a prerequisite, but lacks details on pagination, default behavior, and return format. For a list tool with no output schema, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. Description adds minimal value: only 'text search' reiterates the search parameter. Pagination parameters (page, size) are not explained beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it lists test cases in a project. The text search capability is mentioned, but it does not explicitly differentiate from sibling tool 'search-test-cases' which might have similar functionality.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides a useful prerequisite ('Use list-projects first to get projectId') and mentions text search, but lacks guidance on when to use this tool versus alternatives like 'search-test-cases' or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It does not disclose any behavioral traits such as pagination behavior (despite page and size parameters), idempotency, error handling, or authentication requirements. The description is too brief for transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of two short sentences that convey the core purpose and a key prerequisite. Every word earns its place with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, no output schema, no annotations, and many sibling tools, the description is too minimal. It lacks details on pagination, response format, error conditions, and other essential context for correct invocation. An agent would need additional information to use it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters2/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is low (33%), but the description does not explain the meaning of page and size parameters. It only highlights projectId implicitly by saying to get it via list-projects, but does not elaborate on its format or constraints. The description adds minimal value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List test plans in a project', which is a specific verb+resource. Among sibling tools, it is distinct from other list tools like list-test-cases or list-launches. The resource and action are unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a helpful prerequisite: 'Use list-projects first to get projectId.' However, it does not specify when to use this tool versus alternatives like get-test-plan, nor does it mention when not to use it. The guideline is minimal but useful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the basic purpose but does not mention that it is a read-only operation, whether it has side effects, or how pagination works. For a list tool with no annotations, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, immediately states the purpose, and includes a practical tip. No fluff or redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that there is no output schema and no annotations, the description should explain the return structure, pagination behavior, and endpoint behavior. It only mentions statuses and a usage hint, leaving out crucial details like the format of the returned data or whether pagination applies.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and both 'page' and 'size' have descriptions in the schema. The tool description does not add any additional meaning or usage context beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List', the resource 'workflows', and adds context about statuses. It explicitly mentions statuses (Active, Draft, Deprecated) and how to use the IDs in other tools, which distinguishes it from other list tools like list-test-results or list-projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions a specific use case: using status IDs when setting statusId in create-test-case or update-test-case. However, it does not provide explicit guidance on when to use this tool over alternatives, nor does it state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It notes irreversibility, which is key, but does not disclose side effects on related data (e.g., test results, issues) or cancellation implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no superfluous text. Front-loaded with the action, then irreversibility warning. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete operation with one parameter and no output schema, the description is minimally adequate but lacks information on return value or error handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the description adds no extra meaning beyond the schema's parameter description. Baseline 3 is appropriate as the description does not compensate with additional context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (delete) and resource (test case) with the identifier method (by ID). It distinguishes from sibling tools like create-test-case, update-test-case, and get-test-case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use or when not to use this tool versus alternatives. No mention of required permissions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description only states 'list the values' with no mention of pagination, permissions, or side effects. Minimal behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. Purpose and value are front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations or output schema, the description adequately conveys core function but omits details like output format or required permissions.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have schema descriptions (100% coverage), so baseline is 3. The description adds use case context but does not enhance parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists values for a custom field in a project, and provides a specific use case (building AQL filters). This distinguishes it from siblings like list-project-custom-fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage after getting custom field IDs, but does not explicitly state when to use or when not to use, nor mention alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey all behavioral traits. It implies a read operation but does not disclose authentication needs, rate limits, or what 'basic information' includes. The response format is not described, leaving significant ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences: first states the purpose, second provides a usage alternative. No redundant or extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Lacks an output schema, so the description should compensate by detailing the return data. It only says 'basic information' without enumerating fields, leaving the agent guessing about the response structure.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with a clear description for the only parameter 'id'. The description adds no additional context or format details beyond the schema, so baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Get' and resource 'test case' with scope 'basic information'. It explicitly distinguishes from 'get-test-case-overview' which covers full details, making the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises using 'get-test-case-overview' for full details, providing a clear alternative. Does not address other sibling tools like history or results, but the most relevant comparison is covered.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It discloses the general behavior (retrieving relations) and lists example types (clones, duplicates), but lacks details about output format, permissions, or side effects. Since it is a read operation, the behavior is safe but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence of 12 words, very concise and front-loaded. It could be slightly improved by adding structure, but it efficiently conveys the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema and no annotations, the description is insufficient. It does not explain what the returned relations look like, the format, or the types fully. For a simple tool, more context is needed to guide an AI agent on what to expect.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter 'id' described as 'Test case ID'. The description adds no additional meaning beyond the schema, leading to a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Get') and resource ('relations for a test case') with examples of relation types, clearly distinguishing itself from sibling tools like get-test-case (retrieves the test case itself) and set-test-case-relations (modifies relations).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for retrieving relations but does not explicitly state when to use it over alternatives like get-test-case or set-test-case-relations. However, the naming and context make the usage reasonably clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must disclose behavioral traits. It only states 'Get' which implies a read operation, but it does not specify whether it is read-only, what 'detailed' entails, or any potential side effects or prerequisites. The description is too minimal for full transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at 5 words, front-loaded with the key action and resource. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (1 param, no output schema), the description is somewhat complete but lacks details about the response format or what 'detailed' means. It does not cover error conditions or permission requirements. Adequate but could provide more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with the 'id' parameter described as 'Test result ID'. The description adds 'by ID', reinforcing the parameter's role. However, it adds no additional semantics like format, constraints, or examples beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get', the resource 'detailed test result', and the method 'by ID'. This effectively distinguishes from sibling tools like 'list-test-results' and 'get-test-case'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implicitly conveys usage: retrieve a specific test result when you have its ID. No explicit when-not-to-use or alternatives are provided, but the context with sibling names makes the use case clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the replacement behavior, which is essential. However, it omits other traits like idempotency, required permissions, or potential side effects (e.g., cascading updates). Basic but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with action and key behavior. No filler. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the core operation but lacks context on prerequisites, error handling, and return values. For a mutation tool with no output schema or annotations, more context would help an AI agent avoid mistakes. Adequate but not fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage, so the description does not need to add parameter details. The description does not elaborate on the parameters beyond what the schema provides. Baseline score is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set' and the resource 'requirements for a test case', and adds the critical behavior 'replaces all existing', which distinguishes it from sibling tools like get-test-case-requirements.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage via the replacement behavior, but provides no explicit guidance on when to use this tool vs alternatives (e.g., get-test-case-requirements for reading first, or another tool for incremental updates). This is a missed opportunity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, description carries full burden. It discloses that all existing steps are replaced (destructive), but does not mention reversibility, required permissions, or error states. Minimal but correct.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. Action and key behavioral note front-loaded. Ideal conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, so description should hint at return value; it does not. Also lacks error conditions or test case existence requirement. Adequate for simple update but could be improved.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema describes both parameters with 100% coverage (id as test case ID, steps as array with detailed definition). Description adds no additional meaning beyond schema. Baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states verb 'update' and resource 'scenario (steps) of a test case'. Distinguishes from sibling tools like 'update-test-case' which likely updates other fields, and 'get-test-case-scenario' for read operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for updating steps, but no explicit guidance on when to use vs. alternatives like 'update-test-case' or prerequisites. The note 'Replaces all existing steps' indicates a replacement behavior but not usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It does not mention side effects, permissions, error conditions, or the return value. For a creation tool with many parameters, more transparency is needed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The description is front-loaded with purpose and immediately provides key usage guidance. Highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity and lack of output schema, the description adequately covers what the tool does and what it doesn't do, directing users to sibling tools for advanced functionality. It could mention the response (e.g., returns the created test case) but overall is sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains each parameter. The description adds context by listing some supported fields and advising to use set-* tools for others, but does not significantly enhance understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and the resource 'test case', and it distinguishes this tool from siblings by noting that advanced features like custom fields and issues should be handled by dedicated set-* tools after creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises to create the test case first and then use set-* tools for custom fields, issues, etc., providing clear guidance on when not to use this tool and suggesting alternatives. However, it could be more explicit about prerequisites or specific scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must convey behavior. It indicates a fetch operation with no side effects, but does not disclose any authentication needs, rate limits, or response format. For a simple read, this is minimally adequate but lacks depth.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no redundant or unnecessary words. It front-loads the verb and core object, then provides a clarifying example. Efficient and well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (one parameter, no output schema), the description covers the essential purpose. However, it could be slightly improved by hinting at the response structure, such as 'returns a list of issue links'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter 'id', which the description does not elaborate beyond the schema's 'Test case ID'. No additional meaning is added, meeting the baseline but not exceeding it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves issue links for a specific test case, using a specific verb and resource. The inclusion of 'e.g. Jira issues' clarifies the type of links, making the purpose precise and distinct from write operations like 'set-test-case-issues'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies a read operation but does not explicitly guide when to use this tool over alternatives like 'set-test-case-issues'. No context is provided for prerequisites or exclusions, leaving the agent to infer usage from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as read-only nature, permission requirements, error handling, or performance characteristics. It only states what is returned, which is insufficient for a tool with no annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—two sentences, no wasted words. The first sentence lists what is included, the second gives usage guidance. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no nested objects, no output schema), the description is fairly complete. It explains what the tool returns and when to use it, distinguishing from many sibling tools. However, it lacks behavioral details (error handling, performance) that would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage (the single parameter 'id' is described). The description does not add additional meaning beyond the schema, so it meets the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'FULL test case information', listing specific components (members, issue links, etc.) that distinguish it from sibling tools like get-test-case or get-test-case-members.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description says 'Use this when you need complete test case details', which is a clear usage recommendation, but does not explicitly state when not to use it (e.g., when only specific fields are needed, use the targeted getter).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must fully convey behavior. It discloses the key behavior 'replaces all existing', but lacks details on authentication needs, rate limits, error handling, or what happens with invalid data. Adequate but not thorough.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence that efficiently communicates the core purpose and key detail (replacement). No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 2 parameters and no output schema, the description covers input requirements and the replacement behavior. It lacks edge case details (e.g., empty members array) but is generally complete given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents parameters. The description adds that 'each member needs a user ID and role ID,' which reinforces but does not significantly extend the schema information. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set' and the resource 'members for a test case', and adds that it 'replaces all existing', distinguishing it from potential add/modify operations. This differentiates it from siblings like get-test-case-members and other set-* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for setting members by replacement, but does not explicitly state when to use this tool versus alternatives like get-test-case-members for reading or any potential add/update tool. No guidance on when not to use or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the replacement behavior, a key side-effect. However, with no annotations, lacks info on authorization, idempotency, error handling, or return value. Some transparency but incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise single sentence that front-loads the action and the critical replacement side-effect. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 required params, no output schema), the description covers the essential purpose and the replacement behavior. Could mention success indication or error conditions, but adequate for the context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. Description adds no further semantic value beyond the schema, so baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (set), resource (test keys for a test case), and the replacement behavior. Distinguishes from sibling tools like get-test-case-test-keys and other set-* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage when wanting to replace test keys, but no explicit guidance on when not to use or alternatives like add/remove. Context signals show many sibling tools, but no differentiation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must stand alone. It reveals that the tool returns completion status and test counts, but omits any mention of side effects, rate limits, or required permissions. Since it is a read operation, behavior is mostly implied.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences that efficiently convey purpose and usage. No redundant or unnecessary text; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequate given the simple parameter set and lack of output schema. However, the description could mention the return format (e.g., object with fields) and potential error cases, which would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Only one parameter ('id') exists, and its schema description ('Launch ID') is complete. The tool description adds no extra meaning beyond what the schema provides, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves execution progress of a launch, specifically whether it has finished and test status counts. It distinguishes from sibling tools like 'get-launch' or 'list-launches' by focusing on progress tracking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises using it to differentiate a still-running launch from a completed one. However, it does not mention when not to use it or direct to alternatives for detailed results.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses that it returns runs across launches with specific data fields. However, it does not mention permissions, error handling, pagination behavior beyond what the schema provides, or whether results are sorted. The behavioral disclosure is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no unnecessary words. The first sentence clearly states the action and output, and the second provides a crucial clarification. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description mentions return fields but not in full detail (e.g., data types or formats). It covers the essential distinction from edit logs but could elaborate on why to use this over siblings like get-test-case-overview or get-test-result-history. The description is adequate for a simple read tool but not exhaustive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides 100% coverage of parameter descriptions (id, page, size). The description adds no additional meaning to these parameters beyond the context that the tool retrieves run history for a test case. Following the guideline, baseline 3 is appropriate as the schema already does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it retrieves past runs of a test case across launches, listing specific fields (status, launch, duration, failure message). It distinguishes itself from edit logs and implicitly from sibling tools like get-test-case (current state) by clarifying it is a run history.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes that this is a run history, not an edit log, and states that TestOps does not expose edit changes. This helps the agent avoid misusing the tool for edit history. While it does not explicitly list alternative tools, the negative guidance is clear and useful.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that it returns status, launch, and failure message per run, but does not describe pagination behavior, ordering, or whether only recent runs are included. Some transparency but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core purpose, zero wasted words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given moderate complexity (3 parameters, no output schema, no nested objects), the description covers the main functionality and use case. Missing details on return format and pagination, but the agent can infer from the parameter descriptions. Nearly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all 3 parameters have descriptions). The tool description adds no additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'get', the resource 'previous runs of the same test', and the specific data returned (status, launch, failure message per run). It distinguishes from sibling tools that list test results or get individual results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'Use it to tell a stable failure from a flaky one' provides explicit context on when to use this tool. It does not specify when not to use it or name alternatives, but the use case is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, placing full burden on the description. It discloses that the tool lists custom fields and gives examples, but does not mention behavioral traits such as pagination, performance, or side effects. The description is adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise: two sentences that cover purpose and usage guidance without any fluff. Every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Considering the absence of output schema and annotations, the description provides sufficient context: it identifies the tool's function, provides examples, and links to sibling tools. Minor gaps exist (e.g., not stating the output shape), but it is complete enough for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for all three parameters. The description adds value by explaining the purpose of the output but does not significantly enhance parameter understanding beyond the schema. Baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and the resource 'custom fields configured for a project', with examples like Component, Priority, Team, Story. It is easily distinguishable from sibling tools like list-custom-field-values and list-projects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear context on when to use (list custom fields for a project) and how to use the returned data (IDs for list-custom-field-values, names for AQL filters). While it does not explicitly exclude alternative uses, the guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It indicates a read operation (list) and mentions returned IDs, but does not disclose pagination behavior, rate limits, or output format details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with action, no wasted words. Efficient and direct.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With only 2 parameters and no output schema, the description covers purpose and usage. However, it could mention the return format (e.g., list of objects with id and name) or default pagination to be fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for both parameters (page and size). The description does not add additional meaning beyond the schema, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb (list), resource (test layers), and provides examples (UI, API, Unit). It also indicates how the output is used (setting testLayerId in create/update), which distinguishes it from sibling tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly explains when to use this tool: to obtain IDs for setting testLayerId in other tools. It does not explicitly state when not to use, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses the key behavior 'replaces all existing' which is critical. No annotations, so description carries full burden. Could add more on authorization or validation, but replacement is well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no waste. Front-loaded with action and key behavior. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Covers core purpose and behavior for a simple setter tool. Could mention that it overwrites all existing links, but overall adequate given no output schema and simple params.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline 3. Description adds no extra parameter meaning beyond schema; the replacement behavior is contextual but not parameter-specific.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the verb 'Set', the resource 'issue links for a test case', and the behavior 'replaces all existing'. Distinguishes from siblings like get-test-case-issues by being a mutation that overwrites.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage via examples ('Jira/YouTrack/etc'), but does not explicitly state when to use or not use this tool. Hints at get-test-case-issues via parameter description but not in main text.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description highlights the key behavioral trait of replacing all existing relations, which is critical. However, with no annotations provided, it misses details such as side effects, authorization requirements, what happens with an empty relation array, or idempotency. The description carries the full burden but is not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences, front-loaded with action and behavior, and includes a concise list of types. Every word is necessary, and there is no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool without an output schema, the description covers the essential behavior (set and replace) and valid input types. It could mention prerequisites (e.g., test case existence) or error handling, but given the simplicity, it is fairly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already covers 100% of parameters with descriptions, so the baseline is 3. The description adds the 'replaces all existing' context but does not provide new parameter-level semantics beyond what the schema offers. The listed types are already in the schema enum.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set relations' and the resource 'test case', with the specific behavior 'replaces all existing'. It lists the allowed relation types, distinguishing this from the read-only sibling tool 'get-test-case-relations' and other set tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is used to update relations with a full replacement model, but it does not explicitly mention when to use it versus alternatives like 'get-test-case-relations' to first view existing relations. The clarity of 'replaces all existing' provides some context, but no explicit when-not or alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description must disclose behavior. It explains AQL syntax and deprecation of 'rql', but doesn't mention read-only nature, pagination details (0-based page), or authentication needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise and well-structured: purpose front-loaded, followed by examples, then a helpful tip. No unnecessary verbiage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema; description doesn't indicate what the tool returns (list of test case objects). Given complexity, more context on return format or limitations would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, baseline 3. The description adds significant value by demonstrating AQL syntax and custom field usage, going beyond the schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool searches test cases using AQL, with specific examples. It distinguishes itself from sibling tools like 'list-test-cases' by focusing on query language-based search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides examples and a tip to use other tools for custom field discovery. However, it doesn't explicitly state when to prefer this over 'list-test-cases' or other simple listing tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully carries the transparency burden. It discloses the key behavioral detail that the update merges with existing values rather than overwriting all custom fields. However, it does not mention potential error cases, authorization requirements, or return value, which would be helpful for a complete behavioral picture.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise—just two sentences. The first sentence states the core purpose, and the second provides the crucial usage instruction and behavioral note. Every sentence is necessary and earns its place without any fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 required parameters, no output schema, and numerous siblings, the description adequately covers the purpose, precondition, and merge behavior. It could be improved by mentioning that the update persists immediately or what happens on invalid field IDs, but it is sufficiently complete for an experienced agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema describes each parameter. The description adds value beyond the schema by clarifying the merge behavior and noting that the fields array uses a 'flat format' with customField.id and value id, which helps the agent understand the data structure better.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update custom field values for a test case,' specifying the verb 'Update' and the resource 'test case custom fields.' It distinguishes itself from sibling tools like 'get-test-case-custom-fields' by explicitly referencing it as a prerequisite, and from 'update-test-case' which is more general.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on when to use the tool: after calling 'get-test-case-custom-fields' to know available fields. It explains the merge behavior, helping the agent understand that only specified fields are replaced. However, it does not explicitly state when not to use it, but the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations provided, description carries full burden. It states it is a read operation ('reads') and specifies the content (Gherkin and manual steps). No behavioral traits beyond reading are disclosed, which is acceptable for a simple getter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence that front-loads the purpose. No unnecessary words. Every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Simple tool with one parameter, no output schema. Description fully covers what the tool does and what it returns. No missing information for its complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (parameter 'id' described as 'Test case ID'). Description adds no extra meaning beyond schema. Baseline 3 is appropriate; no added value but schema already sufficient.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'get', the resource 'scenario (steps) of a test case', and distinguishes from siblings by specifying it reads both Gherkin scenarios and manual steps. It differentiates from tools like get-test-case that might retrieve the entire test case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies when to use (to retrieve test case steps) but does not explicitly state when not to use or mention alternatives like get-test-case or get-test-case-overview. Context is clear but lacks exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It mentions that tags, links, and members are replaced entirely ('replaces all'), which is important behavioral detail. It does not explicitly state partial update behavior, but the schema with all optional fields implies it. Overall, it provides good behavioral context for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words. The first sentence states purpose and supported fields, the second directs to alternatives. Perfectly front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has high complexity with 13 parameters and no output schema. The description covers what fields can be updated and provides exclusion guidance. However, it does not describe return values or prerequisites like permissions. Given the complexity, it is adequate but could be slightly richer on behavioral expectations (e.g., partial update confirmation).
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. The description adds little beyond the schema, only listing a subset of fields as examples. It does not provide additional meaning for individual parameters. The description's mention of dedicated tools for certain fields is more about usage than parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing test case and lists specific fields it supports (name, description, tags, links, members). It explicitly distinguishes from sibling tools by noting that custom fields, issues, relations, requirements, and test keys should be handled by dedicated set-* tools. This provides a specific verb+resource and differentiates from alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says when not to use this tool: 'For custom fields, issues, relations, requirements, and test keys — use dedicated set-* tools.' This provides clear context and directs the agent to appropriate alternatives, meeting the highest standard for usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![testops-mcp MCP server](https://glama.ai/mcp/servers/kirchik95/testops-mcp/badges/card.svg)](https://glama.ai/mcp/servers/kirchik95/testops-mcp)

Score Badge

Copy to your README.md:

[![testops-mcp MCP server](https://glama.ai/mcp/servers/kirchik95/testops-mcp/badges/score.svg)](https://glama.ai/mcp/servers/kirchik95/testops-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/kirchik95/testops-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server