Lucius

by ivanostanin

Overview Schema Source Related Servers Score Discussions Admin

Python

Remote

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.15.4
Disambiguation4/5
Most tools target distinct resources (test cases, defects, launches, etc.) with clear boundaries. Minor ambiguity exists between soft delete and permanent archive operations, but descriptions clarify the difference.
Naming Consistency3/5
The majority of tools follow a consistent verb_noun pattern (e.g., create_test_case, list_defects). However, some deviate with longer phrases (e.g., rerun_test_results_manually, link_defect_to_test_case), introducing inconsistency.
Tool Count3/5
With 66 tools, the server is on the heavy side. While the domain (test management) justifies many operations, the count is at the upper limit of reasonable scope, potentially overwhelming for agents.
Completeness4/5
The toolset covers CRUD and lifecycle operations for most entities (test cases, defects, plans, layers, etc.) and includes advanced features like defect matchers and manual sessions. A minor gap is the lack of individual step manipulation within test cases.
Average 3.7/5 across 66 of 66 tools scored. Lowest: 2.8/5.
See the Tool Scores section below for per-tool breakdowns.
- 1 of 1 community issues answered or closed in the last 6 months
- 71 commits in the last 12 weeks
- Last stable release on September 8, 2026
- No critical vulnerability alerts
- No high-severity vulnerability alerts
- No code scanning findings
- CI is passing
This repository is licensed under Apache 2.0.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate it's not read-only or destructive. The description adds no extra behavioral context, such as side effects or authorization needs, so it doesn't add value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short (one sentence), which is concise but lacks structure. It could be expanded with a brief sentence about usage context without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 5 parameters and an output schema, the description is too minimal. It doesn't explain the purpose of the custom field value or how to specify the custom field (id vs name). Incomplete for an agent to use confidently.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description does not add any parameter meaning beyond what the schema already provides (e.g., name, project_id, etc.).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'create' and the resource 'custom field value option', distinguishing it from sibling tools like update and delete. However, it could be more specific about the context (e.g., 'for a specific custom field').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like list_custom_field_values or update_custom_field_value. No prerequisites (e.g., need to specify custom_field_id or name) mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds no behavioral context beyond what annotations already provide (readOnlyHint=false indicates creation). It does not disclose side effects, permissions, or output format, which is important for a creation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence, but it is too terse for a tool with 4 parameters and a complex nested structure. Conciseness is achieved at the cost of completeness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and complex parameters, the description fails to provide sufficient context about the tool's role in test management or how it relates to siblings like link_shared_step. An agent would lack essential selection cues.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage with detailed explanations for each parameter. The description adds no additional meaning, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('reusable Shared Step'). It is specific but does not differentiate from siblings like update_shared_step or delete_shared_step beyond the verb.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., searching or updating shared steps). The description provides no context for selection among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint=false, destructiveHint=false, the description confirms a write operation ('Upload'). However, it adds no additional behavioral context such as idempotency, side effects, or authentication requirements. Annotations are minimal, and the description does not compensate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence with no wasted words. However, it could be slightly expanded to include useful context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having 4 parameters, nested objects, and an output schema, the description is minimal. It does not explain the attachment payload pattern, response format, or any constraints. More context is needed for an AI agent to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds nothing beyond the schema's parameter descriptions. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Upload evidence to a manual test result' clearly states the action (upload) and the target resource (evidence to manual test result). It distinguishes from sibling tools like 'add_test_step_attachment' by specifying 'test result' rather than 'test step', but lacks explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'add_test_step_attachment' or 'upload_test_results'. There is no mention of prerequisites, when not to use it, or context for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description adds a caution about destructiveness, but annotations already declare destructiveHint=true. It does not disclose idempotency, return values, or consequences beyond what annotations provide. Minimal added value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is one sentence with a caution symbol, very concise with no wasted words. However, it could be structured to highlight key points like idempotency or parameter hints.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description does not mention return values (though output schema exists), optional parameters like project_id and output_format, or side effects beyond destructiveness. For a destructive tool with optional parameters, more context would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema describes all 3 parameters with 100% coverage. The description does not mention any parameters or add meaning beyond the schema, so baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a launch by ID', using a specific verb and resource. This distinguishes it from sibling tools like close_launch or delete_test_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines1/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance provided on when to use this tool versus alternatives, prerequisites, or whether to close the launch first. The description is silent on usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true, idempotentHint=true, and destructiveHint=false, making the tool's safety profile clear. The description adds no behavioral context beyond the name, such as pagination default or ordering.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that is front-loaded and contains no unnecessary words. Perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite having an output schema, the description fails to mention the default behavior (e.g., which project is used, pagination defaults, result ordering). For a listing tool with 7 optional parameters, more context would be valuable.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description does not need to explain parameters. Baseline is 3, and the description adds no additional meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'List' and resource 'launches in a project', distinguishing it from sibling tools like create_launch or get_launch. However, it lacks specificity about scope or filtering beyond what parameters provide.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as get_launch for a single launch or list_launch_test_results for test results. No context on prerequisites or typical use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true, so the caution emoji and 'Destructive' are redundant. The description does not mention the required confirm parameter or other behavioral traits like permanent deletion or cascading effects, which are not covered by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short (one sentence) but omits critical details like the confirm parameter requirement. It is concise but under-informative.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, an output schema, and required confirmation, the description fails to mention the need to set confirm=true or the effect of output_format. It is not complete enough for safe usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (all 4 parameters have descriptions). The tool description adds no additional parameter details beyond what the schema provides, scoring baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a test layer from Allure TestOps' uses a specific verb ('Delete') and resource ('test layer'), clearly distinguishing it from siblings like create_test_layer, update_test_layer, and list_test_layers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives (e.g., delete_test_case, delete_test_layer_schema) or any prerequisites. The description only states the action without contextual usage advice.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate mutation (readOnlyHint=false) but description adds no further behavioral context (e.g., async, side effects). Adequate given annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is a single, concise sentence. However, it might be too brief for a tool with 6 parameters; could include slightly more context without bloat.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Does not explain what 'scheduling manual reruns' entails (e.g., triggers immediate execution or queues, impact on existing results). An output schema exists but description doesn't leverage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All parameters have descriptions in the schema (100% coverage). The description adds no extra value beyond what schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it schedules manual reruns for launch results. It distinguishes from sibling tools like submit_manual_test_results by the 'rerun' focus, but lacks explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like submit_manual_test_results or upload_test_results. Missing context on prerequisites or conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description does not disclose any behavioral traits beyond the annotation flags (all false). It fails to mention side effects (e.g., whether a new launch affects existing test runs), required permissions, or the impact of optional parameters like `autoclose` or `external`. With no annotations providing additional context, the description leaves the agent underinformed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no wasted words. It clearly identifies the action and target, making it easy for an AI agent to quickly grasp the tool's primary function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 8 parameters and an output schema, the description is too sparse. It does not explain what a 'launch' represents in Allure TestOps, how it relates to test cases or plans, or how the optional parameters affect behavior. The agent lacks context to use the tool effectively alongside the many sibling tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides descriptions for all 8 parameters (100% coverage), so the description does not need to repeat them. However, the description adds no extra meaning beyond the schema, such as clarifying the relationship between parameters (e.g., how `tags`, `links`, and `issues` interact). A score of 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and the resource ('a new launch'), and specifies the system ('Allure TestOps'). It effectively distinguishes this tool from siblings like `delete_launch`, `close_launch`, and `reopen_launch` by indicating its purpose as creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., `upload_test_results` or `submit_manual_test_results`). It lacks any 'when-to-use' or 'when-not-to-use' context, and does not mention prerequisites or common scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Description adds no behavioral context beyond annotations. Annotations indicate readOnly=false, but no details on permissions, idempotency, or side effects. Description does not disclose any traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, efficiently front-loaded with verb and resource. However, could be slightly more informative given tool complexity, but still concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 13 parameters and availability of output schema, description covers only basic purpose. Lacks context on return value or action outcome, but output schema presumably covers return. Adequate but minimal.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage, so baseline is 3. Description does not add semantic information for parameters beyond what schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'Create' and resource 'test case' with location 'Allure TestOps'. It clearly distinguishes from sibling tools for other operations (e.g., update, delete) and other entities (e.g., defects, test suites).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives. Does not mention prerequisites, context, or exclusions despite many sibling tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Caution about destructiveness adds no value beyond the already present destructiveHint annotation. No disclosure of side effects (e.g., cascade delete, permissions) or required confirmation steps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with clear purpose and warning. Front-loaded but could be slightly more informative without adding length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool, missing context about the confirm parameter (important for safety) and what happens after deletion. Output schema exists but not referenced.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. Description adds no extra meaning to parameters like confirm, project_id, or output_format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb 'Delete' and resource 'test layer schema' directly state the action. Distinguishes from sibling tools that operate on different entities (test cases, test layers, etc.).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternative actions like update or list. Lack of context on prerequisites or when deletion is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds a minor behavioral detail ('including manual execution metadata') beyond annotations. However, it does not disclose pagination, filtering behavior, or error conditions, which are not covered by annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the key action. Every word is necessary, with no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the presence of an output schema and comprehensive parameter descriptions, the tool has 10 parameters and the description is very brief. It lacks context on what test results are, how manual execution metadata relates, or how the various filtering and pagination parameters should be used.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The tool description adds no additional meaning to the parameters; it does not mention any parameter or usage hints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists test results inside a launch, including manual execution metadata. It uses a specific verb and resource, and while it doesn't explicitly contrast with sibling tools, the focus on 'test results' distinguishes it from tools like 'list_test_cases' or 'get_launch'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no when-to-use or when-not-to-use guidance. It does not mention alternatives or context for choosing this tool over similar list tools such as 'list_test_cases' or 'list_defects'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint, idempotentHint, and destructiveHint, so the agent knows it's a safe read operation. The description does not add behavioral insights beyond that (e.g., pagination, filtering behavior). It neither contradicts nor significantly enhances the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no extraneous information. It is concise, though arguably too minimal. Front-loading is adequate.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters and an output schema, the description is quite sparse. It omits context about the hierarchy concept, how the output relates to the tree, or when to use optional parameters. While the schema and output schema fill some gaps, the description could be more informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions, so baseline is 3. The description adds no additional meaning beyond the schema; it does not clarify parameter usage or defaults beyond what is already in the input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List hierarchy suites for a project tree,' clearly identifying the verb (List) and resource (hierarchy suites) with scope (project tree). It distinguishes from sibling tools like create_test_suite and delete_test_suite, though it could be more specific about what 'hierarchy suites' entails.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives, such as when to list all test cases vs. suites. The description does not mention any prerequisites, exclusion conditions, or contextual cues for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With readOnlyHint=false, idempotentHint=false, destructiveHint=false, the description carries the burden but only says 'start a session'. It fails to disclose whether this creates a state, requires permissions, or has side effects beyond initiation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no unnecessary words. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite a descriptive output schema, the description lacks context about prerequisites (e.g., launch state), what a 'manual execution session' entails, or how it relates to sibling tools like close_launch or submit_manual_test_results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 4 parameters have descriptions in the schema (100% coverage), so the description adds no param info. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Start a manual execution session for a launch' is a specific verb+resource phrase that clearly states the tool's action. It distinguishes itself from siblings like create_launch or submit_manual_test_results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, nor does it mention prerequisites or constraints. For example, it doesn't indicate whether the launch must be open or if this tool should precede submit_manual_test_results.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Minimal behavioral disclosure beyond creating a layer. Annotations are non-informative (no readOnly, idempotent, or destructive hints). Description does not state side effects, required permissions, or whether the layer is immediately usable. Context beyond schema is lacking.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no fluff. First sentence states the action clearly; second provides context about what test layers are. Information is front-loaded and easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adquately covers purpose and one parameter hint, but lacks details on when project_id is needed and output_format effects. Output schema exists, so return values are not required. Overall minimal but meets basic needs.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with descriptions. The description reinforces the name parameter with examples but adds no additional meaning to project_id or output_format. Since schema already explains, description adds marginal value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool creates a test layer, defines what test layers are (taxonomy for categorizing test cases), and provides concrete examples ('Unit', 'Integration'). Distinguishes from sibling tools like update_test_layer and delete_test_layer by focusing on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives. Does not mention prerequisites (e.g., requires a project) or when not to use it. Implied usage only from the purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description claims 'CAUTION: Destructive' but annotations set destructiveHint to false, creating a contradiction. This undermines trust; the description should align with annotations. Additionally, it provides some context about shared step expansion at execution time, but the contradiction is severe.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with three sentences: a clear purpose statement, a warning emoji, and a brief explanation of behavior. No unnecessary information; each sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 6 parameters, an output schema, and annotations, the description covers the basic function and a warning, but it does not explain how parameters interact or what to expect in the output. The contradiction with annotations further reduces completeness. The output schema is present but the description doesn't leverage it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 6 parameters. The description adds no additional meaning beyond the schema (e.g., does not clarify 'confirm' requirement or 'position' behavior). Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Link' and the resource 'a shared step to a test case', defining a specific action. It distinguishes from sibling tools like 'unlink_shared_step' by indicating the opposite operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by explaining the function and adding a caution flag, but it does not provide explicit guidance on when to use this tool versus alternatives (e.g., when not to use it, prerequisites, or scenarios to avoid). There is no mention of when linking is appropriate or if there are constraints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, destructiveHint. Description adds no further behavioral context (e.g., pagination, sorting, filtering). No contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One sentence, front-loaded with key information. Could include brief mention of output details but remains concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a list tool, missing details on pagination, sorting, or filtering. Output schema exists but description doesn't elaborate on what is returned.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (one optional parameter). Description does not explain 'output_format' beyond what the schema already provides; no added semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('List'), the resource ('defects'), and the scope ('in the current project'). It differentiates from sibling tools like 'get_defect' (single) and 'list_defect_matchers' (different resource).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'get_defect' for a single defect). No exclusions or prerequisites mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The 'CAUTION: Destructive' note adds limited value beyond the annotations, which already set destructiveHint: true. The description does not elaborate on what 'destructive' entails, nor does it mention side effects, required permissions, or the idempotent nature hinted in annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short (two sentences) and front-loaded. The caution sentence is useful but could be more compact. Every sentence serves a purpose, though the description could be expanded to cover more context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 7 parameters and an output schema, the description is minimal. It does not explain return values, prerequisites, or how to use optional parameters, leaving gaps in completeness despite the presence of an output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no additional meaning to parameters; it only states the tool's purpose without referencing any input specifics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing custom field value', specifying both the verb and resource. It distinguishes from sibling tools like create_custom_field_value and delete_custom_field_value.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for updating existing custom field values but does not explicitly state when to use this tool over alternatives or when not to use it. No exclusions or context for choice are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and idempotentHint=true. The description adds a caution about destructiveness, which is consistent but adds little new information. No behavioral traits beyond annotations are disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise: one sentence plus a caution emoji. Every word earns its place. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite high schema coverage and annotations, the description is minimal and only mentions updating name, while the schema includes other parameters. The caution is the only extra context. Output schema exists but is not leveraged. Overall adequate for a simple update but lacks completeness for all parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions (e.g., confirm: 'Must be set to True to proceed'). The description adds no additional meaning beyond the name parameter being implied. Baseline 3 is appropriate for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description states 'Update an existing test layer's name' which clearly identifies the action and the resource. However, it only mentions name while the schema includes other parameters (project_id, output_format), making it slightly incomplete but still distinct from siblings like create_test_layer or delete_test_layer.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No when-to-use or when-not-to-use guidance is provided. The caution 'Destructive' is generic and does not help distinguish from alternatives like create_test_layer or update_test_layer_schema. No explicit context about prerequisites or scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructive and idempotent behavior. The description adds that currently only the name can be updated, which is a useful limitation, but does not elaborate on side effects, permissions, or other behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences, front-loading the main purpose and adding a clarifying limitation. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks information about return values, error conditions, or required permissions. Even though an output schema exists, the description does not mention what the tool returns or how to handle failures.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% parameter description coverage, so the description adds minimal extra meaning. It reinforces that name is the primary updatable field, but does not explain output_format beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates metadata of a Test Plan and specifically mentions it supports updating the name. This distinguishes it from create, delete, or list operations among siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives, such as when to create a new plan or use other update tools. The description only states what it does without usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate this is a write operation (readOnlyHint=false). Description adds that it targets 'manual attachment steps', but doesn't disclose additional behavioral traits like error handling or step resolution logic.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and target. No unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite high parameter count (10) and complex step resolution logic (step_name, step_index, attachment_id, fixture fallback), the description provides no explanation of how these parameters interact or the expected attachment payload format. Output schema exists but doesn't compensate for missing usage context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema describes all parameters. The description adds no extra meaning beyond the schema's own descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Upload evidence') and the target ('manual attachment step inside a test result'). It distinguishes from similar tools like 'add_test_result_attachment' by specifying step-level attachment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'add_test_result_attachment'). No mention of prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=false and destructiveHint=false, so the description does not need to restate. However, it adds the context of a hierarchy tree, which is helpful. But it doesn't disclose any additional behavioral traits like idempotency, handling of duplicate names, or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, clear sentence with no extraneous words. It is concise and front-loaded. However, it could be slightly more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 5 parameters (1 required) and an output schema exists, the description is minimally adequate. It states the purpose but does not explain the hierarchy concept, return value, or error conditions. With an output schema, the agent can infer return format, but more context would help.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the input schema already describes all parameters. The description does not add any extra meaning beyond what is in the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action ('Create'), the resource ('new test suite node'), and the context ('in the hierarchy tree'). It effectively distinguishes from sibling tools like list_test_suites and delete_test_suite.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., create_test_case, manage_test_plan_content). No exclusions or prerequisites mentioned. The description is purely functional without usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide destructiveHint=true and idempotentHint=true. The description adds a caution emoji but does not elaborate on behavior such as error handling, deletion consequences, or idempotency effects. It adds minimal beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence plus a caution line) and front-loaded. For a simple delete operation, this conciseness is appropriate, though it could include more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the 6 parameters and existence of output schema, the description is minimal. It does not explain post-deletion behavior or potential side effects. However, annotations cover the destructive nature, so it is moderately complete for a simple tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions. The tool description does not add any additional semantic information about parameters beyond what is already in the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb-resource pair 'Delete a custom field value,' clearly identifying the action and object. Among sibling tools, it is distinct from create/update/list operations for custom field values.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., delete_unused_custom_fields or other delete tools). The description only states the action and a caution, lacking context for appropriate usage or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the agent knows this is a safe read. The description adds minimal behavioral context beyond that, mainly the 'summarize' aspect, but does not meaningfully expand on what the agent should expect.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no filler. It communicates the core operation efficiently enough for a simple get-by-ID tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite the vague 'summarize' wording, the tool is adequately contextualized for a read-only retrieval: annotations cover safety, the schema covers all parameters, and an output schema exists. The main missing piece is usage guidance relative to sibling launch tools, but that is already penalized in usage_guidelines.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are already documented. The description adds little beyond implying that launch_id identifies the specific launch to retrieve, which is already clear from the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a clear verb+resource: retrieving a specific launch. The use of 'specific' slightly differentiates it from list_launches, but it does not explicitly name or contrast any sibling tool, and 'summarize its details' is somewhat vague.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
There is no guidance on when to use this tool versus alternatives such as list_launches or get_launch_test_results. The only implicit cue is 'specific launch', but no explicit context, prerequisites, or exclusions are provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description adds minimal behavioral detail beyond what annotations imply (retrieve operation). No contradiction, but no extra context about error behavior or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence (8 words). It is front-loaded but could include a brief note about optional parameters or output format. However, it is not wasteful and fits a simple read tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of output schema, full schema coverage, and annotations, the description is minimally adequate. However, it lacks context about when to use this tool vs. similar custom field tools, which would improve completeness for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all parameters. The description does not add any additional meaning beyond what the schema already provides. Baseline 3 is appropriate since the schema handles parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (retrieve), the resource (custom field values), and the scope (for a specific test case). This differentiates it from sibling tools like get_custom_fields (which retrieves definitions) or list_custom_field_values (which may list all values).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There are many sibling tools related to custom fields (e.g., list_custom_field_values, create_custom_field_value), but no explicit context or exclusions are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true. The description adds 'CAUTION: Destructive' but does not elaborate on the nature of the destruction (e.g., irreversible changes). It adds minimal value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise with two fragments. It is efficient but lacks structure; a single sentence or bullet points would improve readability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (6 parameters, destructive nature) and the presence of an output schema, the description is minimal but sufficient. It covers the core purpose and the caution, but could mention the confirm parameter's role.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter information, meeting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update an existing test layer schema' clearly states the verb (update) and resource (test layer schema), distinguishing it from sibling tools like create, list, and delete test layer schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. There is no mention of prerequisites, when not to use, or which sibling tools to choose instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide idempotentHint=true and destructiveHint=false. Description confirms linking but does not add extra behavioral details (e.g., permissions, scope, or error conditions).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences, no redundant words. Could be slightly more structured, but overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present and full parameter coverage, the description is adequate. It explains the connection between defect and test case, but could mention prerequisites like integration setup.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema handles parameter meaning. Description does not add new semantics beyond what is already in the property descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action ('link a defect to a test case') and the mechanism ('shared issue mapping'), distinguishing it from siblings like 'unlink_issue_from_test_case'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., 'unlink_issue_from_test_case'). The description only mentions benefits but no when-not-to-use or alternative scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the tool's safety profile is clear. The description does not add behavioral context beyond these annotations, but also does not contradict them. It mentions listing but not pagination or filtering specifics, which are covered by the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no wasted words. It is front-loaded and directly states the purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present and full schema coverage, the description is minimally viable but could provide more context about pagination, filtering, or the typical return format. It does not explain the nature of 'available values' further.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 10 parameters with descriptions. The tool description adds no additional parameter-level meaning, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (list) and the resource (available values for a custom field). It effectively distinguishes itself from sibling tools like get_custom_fields which list fields themselves, and create/update/delete custom field values.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as when to list values versus create or update them. No context about prerequisites or typical use cases is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a write operation (readOnlyHint=false), but the description does not add behavioral details beyond the basic action. It does not mention side effects, permissions, or error states. Given annotations exist, the description's contribution is minimal but not misleading.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The main description is only one sentence, which is concise but too brief. It could be more structured to include key usage points without becoming verbose. The schema descriptions compensate somewhat, but the tool-level description is minimal.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness2/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is incomplete given the complexity of the tool. It does not explain prerequisites (e.g., having a test session ID from 'start_manual_test_session'), typical workflow steps, or what the tool returns. The existence of an output schema does not fully mitigate this lack of context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, and the description for the 'results' parameter adds significant value by explaining two submission modes (using result_id or fallback), which is crucial for correct usage. Other parameters are well-described and clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'submit' and the resource 'manual execution results' for a manual session. It distinguishes from sibling tools like 'start_manual_test_session' and 'rerun_test_results_manually' by focusing on submission of results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as 'upload_test_results' (for automated results) or other manual testing tools. The description lacks context on prerequisites or typical workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate the tool modifies data but is not destructive. The description implies appending results but does not add behavioral details beyond the schema, such as permissions or rate limits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. It front-loads the action and resource effectively.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters and an output schema, the description is minimal. It does not mention the maximum results limit (in schema) or return format, but it adequately covers the core operation for a simple upload.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so baseline is 3. The description adds no extra meaning beyond the schema's parameter descriptions, simply restating the core action.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'upload' and the resource 'external test results to an existing launch'. It uses a specific verb and distinguishes from sibling tools that involve manual submission or listing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives like 'submit_manual_test_results'. There is no explicit when/when-not or mention of appropriate contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already include destructiveHint=true. The description adds context by stating 'reassigning test cases changes their hierarchy location,' which is helpful but not extensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences with no wasted words: first states purpose, second provides a crucial caution. Efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of annotations, full schema coverage, and an output schema, the description is adequate. It covers the destructive nature and the core function, though it could expand on hierarchy implications.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema has 100% coverage with descriptions for all 5 parameters. The description does not add additional semantic meaning beyond what the schema provides, so baseline of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the action (assign) and resources (test cases to a suite path). Distinguishes itself from sibling tools like create_test_case or manage_test_plan_content by focusing on hierarchical assignment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool over alternatives. The caution about destructiveness hints at a downside but does not provide decision criteria or compare with related tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and readOnlyHint=false, so the description's 'permanently delete' aligns. However, it does not elaborate on idempotency (idempotentHint=true) or other behavioral traits like potential data loss, so it adds limited value beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence with no wasted words, front-loading the action and resource for quick understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With full schema coverage and an output schema, the description covers the core intent. It could mention the confirm parameter requirement for destructive operations, but overall it provides sufficient context for a bulk deletion tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description does not add extra meaning beyond what the schema provides, meeting the baseline for high coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (permanently delete), the resource (archived shared steps), and the scope (all in current project). This distinguishes it from sibling tools like delete_shared_step (which deletes a single step) and delete_archived_test_cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives (e.g., delete_shared_step for individual deletion), nor does it mention prerequisites or contexts where it should not be used.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate readOnlyHint=true and destructiveHint=false, so the description's value is limited. It adds that validation is performed locally and only requested metadata is synchronized, which provides useful but not critical behavioral context beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, front-loading the main action. It efficiently communicates purpose and key requirements without unnecessary verbiage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With five parameters including enums and an output schema, the description covers the core concept but lacks details on the generated output format, error cases, or specific behavior for edge cases. It is adequate but leaves room for more context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description adds minimal extra meaning (e.g., 'Both target selections are required') but does not significantly enhance understanding beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Generate' and the resource 'test snippet from a TestOps test case', making the tool's primary function immediately obvious. It distinguishes itself from sibling tools like create_test_case or get_test_case_details by focusing on code generation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions that both target selections (language and framework) are required, giving some usage context. However, it does not explicitly state when to use this tool vs alternatives, nor does it provide scenarios where it would not be appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint. The description adds minimal context ('find existing ones') but does not contradict annotations. It provides no additional behavioral details beyond what annotations offer.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, front-loaded with action and resource. No extraneous words. Efficiently conveys purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool is a simple read-only list with pagination and filtering, and full schema coverage plus annotations exist, the description is sufficient for an agent to understand its basic function. An output schema also exists, reducing the need to explain return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so all parameters are documented. The description does not add extra meaning or context for parameters beyond the schema. Baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'list' and resource 'shared steps' with context 'in a project'. It effectively distinguishes from sibling tools like create_shared_step or update_shared_step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives, no exclusions or prerequisites mentioned. Only says 'to find existing ones', which does not differentiate from other listing tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the tool's safety is clear. The description adds context about paginated results and returned fields (IDs, names, tags), but does not reveal any additional behavioral traits beyond what annotations and schema imply. This is adequate but not exceptional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences: the first states the core function, the second adds purpose. No wasted words, front-loaded with key information. It is appropriately sized and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and high parameter coverage, the description is largely sufficient. It explains the tool's purpose and pagination, but could mention default behavior (e.g., returns all cases if no filters applied) or the fact that page/size defaults are 0 and 20 (though already in schema). Overall, complete enough for a list tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description does not add parameter-specific meaning beyond what the schema already provides. It mentions pagination and returned fields but does not elaborate on how parameters like 'tags' or 'status' work (exact match is already in schema). Thus, no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists test cases in a project, specifying the verb 'List' and resource 'test cases'. It mentions the return fields (IDs, names, tags) and pagination. However, it does not explicitly differentiate from the sibling tool 'search_test_cases', which could be seen as a more filtered variant.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises 'Use this to review existing test documentation', giving a general usage context. But it lacks explicit when-not-to-use guidance or mention of alternatives like 'search_test_cases' for complex filtering or 'get_test_case_details' for single case details.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true. The description adds that it returns updated launch details, providing extra context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that is front-loaded with the action and result. No extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Although the tool has an output schema and annotated traits, the description lacks guidance on when to use it among sibling tools. It is adequate but not comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description does not add meaning beyond what the schema already provides. It meets the baseline but does not enhance parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'close' and the resource 'launch', and it adds that it returns updated details, distinguishing it from related tools like delete_launch or reopen_launch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like delete_launch or reopen_launch. The description is minimal and does not provide context for appropriate usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true. The description adds clarity by explicitly stating it is destructive and explaining the confirm parameter as a safety net, which goes beyond the hints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences front-loaded with the action and then the safety requirement. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a delete tool with annotations and output schema, the description covers the basic usage and safety flag. However, it omits what happens if the matcher_id is invalid or if confirm is false (e.g., error vs no-op), which could be important for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents each parameter. The description only mentions the confirm parameter, adding no new information beyond what is in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (permanently delete) and the resource (defect matcher). It distinguishes itself from sibling tools like create_defect_matcher, update_defect_matcher, and list_defect_matchers, and other deletion tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions the destructive nature and the need to set 'confirm' to true, but it does not provide guidance on when to use this tool versus alternatives (e.g., disabling via update_defect_matcher) or any exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. The description simply says 'retrieve', which is consistent but adds no extra behavioral context beyond the annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that states the action and resource. It is front-loaded and efficient, though slightly brief.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple read operation with an output schema present, the description is adequate. However, it lacks usage guidance and could mention output_format options, but the schema compensates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so both parameters are documented in the schema. The description adds no additional meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'retrieve' and resource 'defect', clearly indicating a read operation for a single defect. It distinguishes from siblings like list_defects (listing) and create_defect (creation).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly state when to use this tool versus alternatives like list_defects. Usage is implied by the name and schema but lacks explicit guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, destructiveHint. Description adds that it lists 'all matchers' but does not disclose pagination, sorting, error handling, or other behaviors beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
One concise sentence that front-loads the purpose. Could be slightly more structured (e.g., mentioning output), but it earns its place with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with output schema and clear annotations, the description is fairly complete. It explains the domain (automation rules) but lacks mention of potential error conditions or pagination behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for both parameters. Description adds context by defining matchers as 'automation rules', but adds no further meaning to the parameters beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'List' and resource 'matchers (automation rules)' for a given defect, clearly distinguishing from sibling tools like create/update/delete_defect_matcher.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use versus alternatives (e.g., list_defects or other list tools). Usage is implied by the name and parent defect requirement, but lacks exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations include idempotentHint=true and destructiveHint=true, which the description does not elaborate on. The description adds 'return updated launch details' but omits details about side effects or required permissions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, front-loaded with the main action.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and annotations, the description is adequate but does not mention prerequisites or caution about destructive effects, which would be helpful for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the description adds no additional meaning beyond the schema's parameter descriptions. The baseline of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Reopen'), the resource ('launch'), and the outcome ('return updated launch details'). It distinguishes from sibling tools like close_launch and delete_launch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide when to use this tool vs alternatives, nor does it specify prerequisites (e.g., launch must be closed). Some context is implied but no explicit guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the description's role is minimal. It confirms the read behavior and adds that only test cases currently linked are listed, but doesn't disclose details like pagination limits or rate handling.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no fluff. Every word earns its place, and it avoids unnecessary detail given the rich schema and annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the annotations (read-only, idempotent, non-destructive) and the presence of an output schema, the description is adequate. It could mention pagination or output format defaults, but the schema handles those. The description is sufficient for a simple read operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
All 4 parameters are fully described in the schema with defaults and ranges (100% coverage). The description adds no extra semantic context beyond what the schema provides, so it meets the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'List test cases currently linked to a defect' uses a specific verb ('list') and resource ('test cases') with clear scope ('linked to a defect'), distinguishing it from siblings like 'list_test_cases' (lists all test cases) and 'link_defect_to_test_case' (creates a link).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not explicitly guide when to use this tool versus alternatives. Usage is implied by the name and description, but no exclusions or context are provided, such as when to use 'list_test_cases' instead.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and readOnlyHint=false. The description adds context by specifying the permanence and scope ('permanently delete all archived/deleted test cases'). It does not contradict annotations. However, it could warn about the lack of recovery or mention that the confirm parameter safeguards the action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, well-structured sentence that front-loads the core action. No unnecessary words or information, making it highly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple destructive bulk action, annotations (destructiveHint), input schema, and presence of an output schema, the description covers the essential behavioral and usage context. No gaps are apparent for this tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and the description adds no additional explanation for parameters. The schema already defines confirm, project_id, and output_format with descriptions. Baseline 3 is appropriate as the description does not enrich parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool permanently deletes all archived/deleted test cases within the current project. It uses a specific verb ('delete') and resource, distinguishing it from siblings like delete_test_case (single) or delete_archived_shared_steps.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It does not mention that this is for bulk cleanup of archived items, nor does it exclude scenarios where delete_test_case is more appropriate. Sibling tools exist for single deletion, but no comparison is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveness, but the description adds specific warnings about breaking references in test cases. This provides valuable behavioral context beyond what annotations convey, without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with two sentences and two caution notices. It is front-loaded with purpose and wastes no words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and full parameter descriptions, the description sufficiently covers purpose and key side effects. It is complete for the tool's complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the description does not need to add parameter details. It does not provide additional meaning beyond the schema, earning a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Delete', the resource 'shared step', and the location 'library'. It distinguishes the tool from siblings like 'delete_test_case' and 'delete_archived_shared_steps'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context for when to use (to delete a shared step) but lacks explicit guidance on when not to use it or alternatives. The caution about breaking references implies careful usage but no direct comparison to other tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide destructiveHint=true and idempotentHint=true. The description adds context by stating 'Permanently removes' and explaining idempotent behavior ('if the plan does not exist, it returns success'), enhancing transparency beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the action, uses a caution emoji for emphasis, and contains no unnecessary words. Every sentence adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple deletion tool with a safety confirm parameter, the description covers core behavior, permanence, and idempotency. It does not mention permissions or cascading effects, but given the tool's simplicity and presence of an output schema, it is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with all three parameters described. The description does not add parameter-specific information beyond the schema; e.g., it doesn't mention the confirm safety flag. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a Test Plan' with the verb 'Delete' and resource 'Test Plan'. It further clarifies with 'Permanently removes the Test Plan from Allure TestOps', distinguishing it from sibling tools like delete_test_case or delete_launch.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a caution about destructiveness and mentions idempotency, but does not explicitly state when to use this tool versus alternatives (e.g., manage_test_plan_content, update_test_plan) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint, indicating a safe read operation. The description adds context about the mapping between custom fields and test layers, which is valuable beyond the annotations. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, using only two sentences to convey the purpose and a brief explanation. Every sentence adds value with no extraneous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and comprehensive parameter descriptions, the description covers the essential aspects. It explains the concept of test layer schemas, which adds completeness. Missing explicit usage guidelines slightly reduces the score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with detailed parameter descriptions. The description does not add further meaning to the parameters beyond what the schema already provides, so it meets the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it lists test layer schemas for a project, with an explanation of what these schemas are. It distinguishes itself from sibling tools like 'create_test_layer_schema' or 'list_test_layers' by focusing on schemas that map custom fields to layers.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving mapping information but does not explicitly state when to use this tool instead of alternatives like 'list_test_layers' or provide context such as prerequisites or typical workflow scenarios.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark destructiveHint true; description reinforces with 'CAUTION: Destructive' and explains effect on execution. Adds context beyond annotations about the runtime impact.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Extremely concise. One sentence plus caution and explanation. Every word contributes; no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Adequately covers purpose and effect. Does not mention error conditions or prerequisites, but schema is rich and output schema exists. Sufficient for an agent to understand the tool's basic function.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so baseline is 3. Description does not add additional parameter-level meaning beyond what schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states it removes a shared step reference from a test case. The verb 'remove' and resource 'shared step reference' are specific. Distinguishes from siblings like 'link_shared_step' and 'delete_shared_step'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implied usage: to unlink a shared step from a test case. But no explicit when-to-use or alternatives. Does not compare to similar tools like 'delete_shared_step' or explain when to choose this over other operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true and idempotentHint=true. The description adds that at least one field is required, but does not elaborate on specific behaviors like error conditions or result format. With annotations present, the bar is lower, but some added context would improve transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at two sentences with no redundant information. It is front-loaded with the action 'Update', making it easy for an agent to quickly grasp the tool's purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (5 parameters, output schema exists) and the presence of annotations, the description is fairly complete. It covers the essential purpose and constraint. However, a brief note about idempotency or the effect on existing data could enhance completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with each parameter having a description. The description only mentions 'name or regex patterns', which aligns with the schema but adds no new meaning beyond what the schema already provides. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'update' and the resource 'defect matcher', specifying that it modifies name or regex patterns. This distinguishes it from siblings like create_defect_matcher and delete_defect_matcher.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description notes that at least one field must be provided, giving a clear usage condition. However, it does not explicitly state when not to use this tool or compare it to alternatives like create_defect_matcher for new matchers.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate the tool is not read-only, idempotent, or destructive. The description adds no further behavioral details (e.g., error handling, permissions, conflicts). With annotations present, the bar is lower, but description could still be more informative.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise: two paragraphs sum up the purpose and function without redundancy. Every sentence contributes to understanding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (4 parameters, no nested objects, existence of output schema), the description adequately explains the concept of test layer schemas and their role. It provides sufficient context for an agent to decide to use this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all parameters. The description repeats the general concept (mapping key to layer) but does not add specific parameter-level semantics beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new test layer schema that maps a custom field key to a test layer. It distinguishes from sibling tools like create_test_layer (which creates layers) and other schema tools (list, update, delete) by focusing on the creation of mappings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that schemas allow automatic assignment of test cases to layers based on custom fields, giving context for when to use this tool. However, it lacks explicit statements about when not to use it or alternatives (e.g., if schema already exists).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Discloses soft delete nature and recovery option, adding context beyond destructive annotation. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Concise with front-loaded action, though caution messages are slightly repetitive; overall efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With output schema present, description adequately covers behavior and side effects; could mention idempotency hinted by annotation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers all parameters with descriptions; description adds no additional parameter-specific meaning, meeting baseline for 100% coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clearly states the tool archives an obsolete test case via soft delete, distinguishing it from siblings like delete_archived_test_cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Describes when to use (obsolete test cases) and effects on historical data, but lacks explicit comparison to alternatives like delete_archived_test_cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare destructiveHint=true and readOnlyHint=false, and the description reinforces this with an explicit CAUTION warning. It adds the useful caveat that nested entities are handled according to Allure TestOps's tree-group API behavior, though it stops short of explaining exactly what happens to those nested items. No contradiction with the annotations exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is short and front-loaded: the core action and a danger warning appear in the first two lines, followed by the nested-entity caveat. There is no filler or repetition, though the final sentence is somewhat vague and could be more specific without hurting conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given a destructive tool with detailed per-parameter schema descriptions and an output schema, the description is reasonably complete: it states what is removed, warns about destructiveness, and flags nested-entity behavior. It does not need to repeat the confirm flag or output format details already in the schema. The main gap is the unspecified effect on child nodes, but the API-behavior caveat at least acknowledges this nuance.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all five parameters, including the confirm safety measure and optional tree_id/project_id. The description adds only a slight reinforcement that suite_id targets a suite/group node, but does not need to explain parameter semantics further. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Delete a test suite node from hierarchy.' It further defines the target as a 'suite/group node,' which distinguishes it clearly from sibling deletion tools like delete_test_case, delete_test_plan, and delete_launch. The purpose is immediately identifiable and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description establishes clear context: this tool is for removing hierarchy suite/group nodes, so it is appropriate when deleting a test suite node rather than a test case, plan, launch, or shared step. It does not explicitly name alternatives or when-not conditions, but the resource scoping is sufficient for correct routing in most cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint=true, idempotentHint=true), description adds the specific 'unused' condition and scope 'in the current project'. No contradictions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Single, clear sentence. No waste, front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Tool has output schema, so return values are covered. Description covers purpose, condition, and scope. Complete for this use case.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so description adds little beyond schema's parameter descriptions. Description does not elaborate on confirm or output_format beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the action (delete), resource (custom fields), and condition (unused by any test case). It distinguishes from sibling tools like delete_custom_field_value or delete_test_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Implies usage for removing orphaned custom fields but lacks explicit when-to-use or when-not-to-use guidance, and does not mention alternatives like get_custom_fields or delete_custom_field_value.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, destructiveHint, so the description carries low burden. It adds minimal behavioral context beyond stating the tool lists integrations, which is consistent with the annotations. No extra details on rate limits or auth are provided, but the annotations cover the key traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two concise sentences with no wasted words. The purpose and usage are front-loaded, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the output schema exists (not shown but stated) and the tool is simple, the description sufficiently explains what the tool does and why to use it. No additional context is needed for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with both parameters having descriptions. The tool description does not add further parameter information, so a baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists available integrations (issue trackers) in Allure TestOps and explains the purpose: to discover which integrations are configured for linking issues. It distinguishes itself from sibling tools by being the only integration-listing tool.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly guides when to use: 'Use this tool to discover which integrations ... are configured and available.' While it doesn't mention when not to use, there are no alternative tools for this task among siblings, so the guidance is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, and destructiveHint, establishing the tool as safe and read-only. The description adds context about the taxonomy purpose but no additional behavioral traits beyond what annotations convey. This is adequate but not enhanced.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise with three sentences across two paragraphs. Every sentence adds value: first states action, second explains purpose, third gives usage guidance. No redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description need not detail return values. It adequately explains the taxonomy concept and provides sufficient context for using the tool. With annotations and output schema, the description is complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, and each parameter has a clear description. The tool description does not add any extra meaning beyond the schema, only referencing output (layer IDs and names). Per guidelines, baseline 3 is appropriate when schema covers parameters well.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool lists test layers to discover available taxonomy. It explains the purpose (categorizing test cases) and explicitly notes it is used to find layer IDs and names before creating or updating test cases, distinguishing it from sibling tools like create_test_layer, update_test_layer, and list_test_layer_schemas.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states 'Use this to find layer IDs and names before creating or updating test cases,' providing clear context for when to use the tool. It does not specify when not to use or name alternatives, but the sibling set includes related tools, making the usage guidance clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive. Description adds pagination behavior and return fields (IDs, names, test case count), providing useful behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with purpose, no unnecessary words. Efficient and clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, input schema fully described, annotations cover safety. Description adds pagination and return field info. Complete for agent usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear parameter descriptions. Description adds no new parameter details beyond mentioning pagination, which is already implied by page/size.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states action (List), resource (Test Plans), and scope (current project). It distinguishes from sibling tools like list_test_cases or list_defects.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use or when-not-to-use guidance. Only implies project context. Does not mention alternatives or exclusion criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructiveHint=true, and the description adds useful context: the operation is permanent and affects both the defect and associated matchers. There is no contradiction with annotations. The description does not cover all behavioral nuances (e.g., error handling), but it provides meaningful additional information.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise at two sentences, with no wasted words. It front-loads the core action and immediately follows with the critical usage condition, making it efficient for an agent to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low tool complexity and comprehensive schema and annotations, the description is largely complete. It covers the destructive nature and the required confirmation flag. The presence of an output schema alleviates the need to explain return values, but a brief note on prerequisites or error cases would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so the baseline is 3. The description emphasizes the importance of the 'confirm' parameter, which is already documented in the schema, but does not add new meaning for the other parameters ('defect_id' and 'output_format').
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: permanently delete a defect and all its associated matchers. This is a specific verb+resource combination that distinguishes it from siblings like 'delete_defect_matcher' (only deletes a single matcher) and 'update_defect' (modifies instead of deletes).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on the required condition: the 'confirm' parameter must be set to 'true' to actually perform the deletion. However, it does not explicitly mention when not to use this tool or suggest alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover the read-only, idempotent, and non-destructive nature. The description adds valuable behavioral context beyond annotations: attachments are not directly returned but require a separate prepare_attachment_download call followed by an HTTP GET to a Lucius URL. This is important for correct usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and front-loaded with the core purpose. The attachment workflow sentence earns its place. The opening sentence is slightly redundant with the title, but overall the structure is clear and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With an output schema present, full parameter schema coverage, and annotations describing the safety profile, the description covers the essential behavior, attachment handling workflow, and a relevant use case. Nothing critical is missing for an agent to invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters. The description does not add much parameter-level detail beyond clarifying that test_case_id is also needed in the attachment download workflow. This meets the baseline but does not exceed it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves complete details of a specific test case, listing the included components (steps, tags, custom fields, attachments). This distinguishes it from sibling tools like list_test_cases, get_test_case_custom_fields, and prepare_attachment_download.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to use this before updating a test case to understand its current state, and it gives a concrete follow-up workflow for downloading attachments. It provides clear context but does not explicitly state when not to use it or compare it to alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate destructive (destructiveHint=true) and idempotent behavior. The description adds value by enumerating the three types of modifications (add, remove, update filter), which is beyond what annotations provide. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the core action. Every sentence adds value with no fluff. The structure efficiently conveys the tool's purpose and capabilities.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, output schema present, annotations), the description covers the main operations. However, the 'or' in 'adding or removing... or updating' could be misinterpreted as mutually exclusive, whereas the schema allows combining operations. Minor ambiguity keeps it from being a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions for each parameter. The tool description groups the parameters into operations but does not add new constraints or details beyond the schema. Baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Modify the content (Test Cases) of an existing Test Plan' and lists specific operations: adding/removing Test Cases by ID or updating the AQL filter. This distinguishes it from sibling tools like update_test_plan which likely handles other attributes.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates when to use the tool (modifying test plan content) but does not explicitly mention alternatives or when not to use it. The specificity of operations provides implicit guidance, but explicit exclusion of other update tools would improve clarity.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Beyond annotations (destructiveHint, idempotentHint), the description adds critical context: changes propagate to ALL test cases, only provided fields are updated, and confirm must be True. This significantly informs the agent's usage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, front-loaded with the action, and uses warnings effectively. Every sentence adds value with no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description adequately covers key behaviors: propagation, idempotency, partial updates, and safety confirm. It lacks information about error handling or required permissions but is generally sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with clear descriptions. The description adds minimal extra meaning, only reinforcing the confirm parameter's role as a safety measure. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update an existing shared step,' specifying a concrete action and resource, distinguishing it from sibling tools like create_shared_step, delete_shared_step, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides clear warnings about destructive impact and propagation to all test cases, emphasizes the need for confirm=True, and notes idempotency. While not explicitly contrasting with alternatives, the guidance is strong.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate a write operation (readOnlyHint=false, destructiveHint=false) align with 'create'. Description adds context about linking to test results but doesn't disclose missing details like duplicate handling or project scope. With annotations present, the description is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences front-load the purpose and provide usage context without unnecessary words. Efficient for a tool with few parameters.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's core responsibility and usage context. With an output schema present, no need to detail return values. Sufficiently complete for a create operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so description adds no new parameter details beyond the schema. The description mentions 'known defect' but doesn't elaborate on parameters like name, description, or output_format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new defect in the current project and explains its purpose for linking to failing test results via defect matchers. It distinguishes from sibling tools like update_defect, list_defects, and create_defect_matcher.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides a clear usage scenario (registering a defect for future linking) without explicitly stating when not to use or listing alternatives. The context is sufficient given the sibling set.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already provide readOnlyHint, idempotentHint, and destructiveHint, so the description's job is lighter. It adds that the tool returns 'allowed values,' which is not in annotations or schema. However, it does not disclose other behavioral traits like data source or limitations. This is adequate but not exceptional.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of two concise sentences. The first states the core purpose, and the second provides usage guidance. Every sentence adds value, with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and annotations, the description provides sufficient context for the agent: it explains the tool's role in the workflow (discovery before creation/update) and its input/output nature. No critical gaps exist for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Input schema covers all 3 parameters with full descriptions, achieving 100% coverage. The description does not add new parameter semantics beyond what is in the schema. According to rules, baseline is 3 when coverage is high, and no extra value is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool retrieves available custom fields and their allowed values for a project. It uses specific verbs and resources ('Get available custom fields') and differentiates itself from sibling tools like list_custom_field_values and create_custom_field_value by emphasizing it is used to discover fields before creating or updating test cases.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool: 'Use this tool to discover what custom fields are available... essential before creating or updating test cases.' This provides clear context and implies when not to use it (when not needing discovery). It also hints at the prerequisite nature for other operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, covering the safety profile. The description adds useful behavioral context by clarifying that this is a targeted, non-recursive read and that it only returns directly assigned test cases, which is beyond what the annotations or schema alone convey.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no wasted words. The core purpose is front-loaded, and the second sentence adds only the scoping details needed to avoid misuse with broader read tools.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple, read-only, idempotent tool, the description combined with the fully described schema and existing output schema is complete. The agent has everything needed to call it correctly: required parameter, optional parameters, output format, and behavioral boundaries.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all five parameters. The description does not add meaning about parameters beyond what the input schema already provides; therefore the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Get test cases directly assigned to one test suite.' It distinguishes itself from broader enumeration and recursive traversal by calling itself a 'targeted hierarchy read,' which differentiates it from sibling tools like list_test_cases and list_test_suites.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use it: when the agent needs only test cases directly assigned to a single suite. It also states exclusions—does not enumerate unrelated suites or recursively traverse the project tree—but it does not explicitly name an alternative tool for recursive traversal, so it falls just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint=true, idempotentHint=true, destructiveHint=false. Description adds behavioral details: case-insensitive name search, exact tag match, combined queries, and the precedence of 'aql' over 'query'. No contradictions with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Well-structured with clear sections and examples. Slightly repetitive with AQL reference appearing both in description and schema. Efficient overall.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 6 parameters, full schema coverage, and an existing output schema, the description adequately covers search modes, parameter interactions, and external references. Complete for use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but description adds extra context beyond schema: explains simple query syntax with examples, clarifies interaction between 'query' and 'aql' parameters. Baseline is 3; description earns a 4.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Clear verb 'Search' and resource 'test cases' with explicit search methods (name, tag, AQL). Distinguishes from siblings like 'list_test_cases' and 'get_test_case_details'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explains two search modes (simple query vs AQL) with syntax examples and a link to AQL docs. Notes that 'query' is ignored when 'aql' is provided. However, does not explicitly contrast with 'list_test_cases' for unfiltered listing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate the tool is not read-only, not idempotent, and not destructive. The description adds behavioral context beyond annotations by explaining how matchers work (matching error messages/stack traces) and the requirement for at least one regex, which is a behavioral constraint not captured in the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description consists of three concise sentences that are front-loaded: first states the main action, second explains the mechanism, and third gives a key constraint. Every sentence adds essential information with no unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, 2 required, no enums) and the presence of an output schema, the description adequately explains the tool's function and key constraints. It covers the core behavior and the important 'at least one' rule, which is sufficient for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 100% schema description coverage, the baseline is 3. The description adds value by explicitly stating the business rule that at least one of message_regex or trace_regex must be supplied, and by providing context on how these regex patterns are used (matching error messages or stack traces).
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and the resource 'defect matcher', and explains its purpose as an automation rule that links future failing test results to a defect based on regex patterns. This distinguishes it from siblings like create_defect or update_defect_matcher.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool: to set up automatic linking of test failures to a defect. It also explicitly states the constraint that at least one regex must be supplied. However, it does not explicitly mention alternatives or when not to use it, but the naming and sibling list make it clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, and non-destructive. The description adds meaningful context beyond that: the result is non-recursive, attachment references identify their verified owner, and downloads require calling prepare_attachment_download and using the returned Lucius URL before expiry. This is valuable behavioral disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the core purpose. The second sentence adds an actionable attachment workflow that is directly relevant. No filler is present, though the second sentence is dense.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the annotations, full schema coverage, and output schema, the description covers the essential behavior, scope exclusion, and attachment handling without missing anything needed for correct invocation. The tool is simple and the description completes it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the schema already explains test_result_id, project_id, and output_format precisely. The description adds no additional parameter-level meaning, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Retrieve') and a specific resource ('one complete TestOps result'), and adds an explicit scope limitation ('without recursively reading related results'). This distinguishes it from sibling list/search tools and makes the tool's purpose immediately clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The first sentence gives clear context for when to use this tool: retrieving a single complete result, not recursively related results. The attachment guidance also routes the agent to prepare_attachment_download when needed, though it does not explicitly name alternatives for retrieving multiple results.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate readOnlyHint=false and destructiveHint=false, which are consistent with creating a non-destructive entity. The description adds behavioral context by explaining that the tool can create a plan using explicit IDs, an AQL filter, or both. This goes beyond the annotations without contradiction.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three sentences, front-loaded with the primary action and resource. It efficiently covers purpose and methods without unnecessary words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity and the existence of an output schema, the description provides sufficient context: it identifies required parameter (name), explains optional parameters, and outlines the two content selection strategies. No gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds value by explaining the dual approaches (test_case_ids and aql_filter) and their combined use, enhancing understanding beyond the schema. Output_format is well-described in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and the resource 'Test Plan in Allure TestOps'. It differentiates from siblings like 'manage_test_plan_content' and 'update_test_plan' by focusing on creation, and explains the two primary methods for defining content (test_case_ids and aql_filter).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use the tool (to create a new test plan) and outlines the two content selection methods. It does not explicitly state when not to use it or name alternatives, but the context implies it is for creation only, which is clear from the purpose.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint and idempotentHint as true. The description adds useful behavioral details like case-insensitive matching and partial match acceptance, which go beyond annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two sentences, covering all necessary aspects without verbosity. It effectively prioritizes key information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a read-only tool with an output schema, the description is complete. It covers both usage modes (single project retrieval and listing) and provides sufficient context for an agent to use the tool effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining the name parameter's matching behavior (case-insensitive, partial match) and clarifying the output format default ('json' instead of null). This provides extra context beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool retrieves a specific project by name or lists all accessible projects. This distinguishes it from sibling tools like create_test_case or get_test_case_details, which operate on different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description explains when to omit the name parameter (to discover project IDs) and details name matching behavior (case-insensitive, exact match preferred). However, it does not explicitly mention when not to use this tool or provide alternatives, but context makes it clear as a read-only tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already mark this as read-only and idempotent, so the description adds valuable non-obvious behavior: the returned URL is one-time and short-lived, must be GET before expires_at without an Allure bearer token, and preparation should be repeated if the URL expires or was already retrieved. This is exactly the kind of behavioral context an agent needs.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Three compact sentences carry the full workflow: the action, the prerequisite call sites, and the URL expiry/recovery behavior. Every sentence contributes essential information with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complete schema, the presence of an output schema, and annotations that establish safety, the description covers everything else needed for correct invocation: prerequisite context, one-time URL handling, auth requirements, and failure recovery. No important call-time behavior is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents each parameter. The description adds mild context by mentioning 'attachment ID, kind, and owner context,' but it does not materially enrich the parameter semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Prepare one verified attachment download, then HTTP GET its returned Lucius URL.' It clearly identifies the tool's two-step purpose and differentiates it from upload-focused siblings like add_test_result_attachment and retrieval tools like get_test_result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to call this tool: 'Call this after get_test_result or get_test_case_details provides an attachment ID, kind, and owner context.' It also advises when to re-invoke the tool after expiry or prior retrieval. It does not name alternatives or exclusions, so it falls just short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description states the operation is idempotent, aligning with the idempotentHint annotation. It explains the behavior when the issue is already unlinked (returns successful confirmation). This adds context beyond the annotations without contradicting them.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise with two short paragraphs. The first sentence clearly states the purpose, and the second adds necessary behavioral and parameter context. No superfluous words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the presence of an output schema, and the annotations, the description provides sufficient context. It covers idempotency, input format, and result behavior, making it complete for invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, but the description adds value by explaining that the issue_id can be either a project key (e.g., PROJ-123) or an internal issue-link ID. This clarifies the parameter's meaning beyond the schema's description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Unlink an issue from a test case,' which is a specific verb+resource pair. It distinguishes itself from sibling tools like 'link_defect_to_test_case' and other test case management tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description mentions idempotency and the acceptable formats for the issue ID (key or internal ID), providing clear context for usage. However, it does not explicitly state when to use this tool versus alternatives or provide exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations indicate destructiveHint=true and idempotentHint=true, so the description doesn't need to reiterate that. It adds value by explaining null handling: 'Fields set to null/None are left unchanged.' This clarifies idempotent behavior. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, no wasted words. The first sentence states the purpose and scope; the second provides a critical usage rule. Perfectly front-loaded and efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and 100% schema parameter coverage, the description covers all necessary context: what the tool does, required fields, and null behavior. No gaps remain for an AI agent to misunderstand.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so parameters are already well-documented. The description adds a behavioral constraint not present in the schema: at least one field must be provided beyond defect_id. This compensates for the schema's lack of that constraint.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update'), the resource ('existing defect'), and the specific fields that can be updated (name, description, or status). This distinguishes it from siblings like create_defect, get_defect, and delete_defect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a key usage condition: 'At least one field must be provided.' It doesn't explicitly state when to use this tool versus alternatives, but the condition is helpful. A more explicit comparison or exclusion would warrant a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds value beyond annotations by explaining destructive nature, partial update, and that steps/attachments replace existing ones. No contradiction with annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Short and focused: states purpose, caution, and key behaviors in a few lines. No extraneous information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given output schema exists, description covers all necessary aspects: partial update, destructive, idempotent, and replacement semantics. Complete for a complex tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (baseline 3). Description adds context about partial update behavior and confirms safety measure, going beyond schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Update an existing test case in Allure TestOps' with a specific verb and resource. It distinguishes from siblings like create_test_case and delete_test_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides guidance on partial update behavior and when to omit fields to preserve values. Implicitly differentiates from create/delete but does not explicitly list alternative tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Lucius MCP server – quality and maintenance score on Glama

Copy to your README.md:

[![Lucius MCP server – quality and maintenance score on Glama](https://glama.ai/mcp/servers/ivanostanin/lucius-mcp/badges/card.svg)](https://glama.ai/mcp/servers/ivanostanin/lucius-mcp)

Score Badge

Copy to your README.md:

[![Lucius MCP server – quality and maintenance score on Glama](https://glama.ai/mcp/servers/ivanostanin/lucius-mcp/badges/score.svg)](https://glama.ai/mcp/servers/ivanostanin/lucius-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/ivanostanin/lucius-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server