TestRail MCP Server
Enables comprehensive interaction with TestRail's API for managing test projects, suites, cases, runs, plans, results, milestones, attachments, and user management. Supports full test lifecycle operations including creating/updating test cases, executing test runs, adding results, and managing test plans.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@TestRail MCP Servershow me the test cases for project 123 in the regression suite"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
TestRail MCP Server
A Model Context Protocol (MCP) server for seamless TestRail integration. This server enables AI assistants like Claude, Cursor, and other MCP-compatible clients to interact directly with your TestRail instance.
Features
🔐 Secure Authentication - API key-based authentication with TestRail
📊 Full TestRail API Coverage - Support for TestRail API v9.7.2
🚀 Easy Integration - Works with Cursor, Claude Desktop, and other MCP clients
📦 Zero Configuration - Run directly with
npx
Related MCP server: Testmo MCP Server
Installation
Global Installation
npm i -g @tenbarrel6/testrail-mcpLocal Installation
npm i @tenbarrel6/testrail-mcpUsing npx
No installation required - run directly in downloaded github repository:
npx @tenbarrel6/testrail-mcpConfiguration
Environment Variables
Create a .env file in your project root or set environment variables:
TESTRAIL_URL=https://your-domain.testrail.io
TESTRAIL_USERNAME=your-email@example.com
TESTRAIL_API_KEY=your-api-key-hereCursor IDE Configuration
Add to your Cursor settings (.cursor/mcp.json or global settings):
{
"mcpServers": {
"testrail": {
"command": "npx",
"args": ["-y", "@tenbarrel6/testrail-mcp"],
"env": {
"TESTRAIL_URL": "https://your-company.testrail.io",
"TESTRAIL_USERNAME": "your-email@company.com",
"TESTRAIL_API_KEY": "your-api-key-here"
}
}
}
}Claude Desktop Configuration
Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):
{
"mcpServers": {
"testrail": {
"command": "npx",
"args": ["-y", "@tenbarrel6/testrail-mcp"],
"env": {
"TESTRAIL_URL": "https://your-company.testrail.io",
"TESTRAIL_USERNAME": "your-email@company.com",
"TESTRAIL_API_KEY": "your-api-key-here"
}
}
}
}Getting Your TestRail API Key
Log in to your TestRail instance
Go to My Settings (click your name in the top right)
Navigate to the API Keys tab
Click Add Key to generate a new API key
Copy and save the key securely
Available Tools
Projects
Tool | Description |
| Get all TestRail projects |
| Get a specific project by ID |
Test Suites
Tool | Description |
| Get all test suites for a project |
| Get a specific test suite by ID |
| Create a new test suite |
| Update an existing test suite |
Sections
Tool | Description |
| Get all sections for a project/suite |
| Get a specific section by ID |
| Create a new section |
| Update an existing section |
| Delete a section |
| Move a section to another parent or position |
Test Cases
Tool | Description |
| Get test cases for a project/suite |
| Get a specific test case by ID |
| Create a new test case |
| Update an existing test case |
| Delete a test case |
| Delete multiple test cases |
| Copy test cases to another section |
| Move test cases to another section |
| Get all available test case types |
| Get all available test case fields |
| Get the edit history for a test case |
Test Runs
Tool | Description |
| Get test runs for a project |
| Get a specific test run by ID |
| Create a new test run |
| Update an existing test run |
| Close a test run |
| Delete a test run |
Tests
Tool | Description |
| Get tests for a test run |
| Get a specific test by ID |
Results
Tool | Description |
| Get results for a test |
| Get results for a test case in a run |
| Get all results for a test run |
| Add a test result |
| Add a test result for a specific case in a run |
| Add multiple test results by test IDs |
| Add multiple test results for cases in a run |
Test Plans
Tool | Description |
| Get test plans for a project |
| Get a specific test plan by ID |
| Create a new test plan |
| Update an existing test plan |
| Close a test plan |
| Delete a test plan |
| Add test runs to a test plan |
| Update a test plan entry |
| Delete a test plan entry |
| Add a test run to an existing plan entry |
| Update a test run inside a plan entry |
| Delete a test run from a plan entry |
Milestones
Tool | Description |
| Get milestones for a project |
| Get a specific milestone by ID |
| Create a new milestone |
| Update an existing milestone |
| Delete a milestone |
Users
Tool | Description |
| Get all users (optionally filtered by project) |
| Get a user by ID |
| Get a user by email address |
| Get the current authenticated user |
Statuses & Priorities
Tool | Description |
| Get all available test result statuses |
| Get all available test case statuses (Enterprise) |
| Get all available test case priorities |
Templates & Configurations
Tool | Description |
| Get all templates for a project |
| Get all configurations for a project |
| Get all available result custom fields |
Attachments
Tool | Description |
| Add an attachment to a test case |
| Add an attachment to a test result |
| Add an attachment to a test run |
| Add an attachment to a test plan |
| Add an attachment to a test plan entry |
| Get/download an attachment by ID |
| Get all attachments for a test case |
| Get all attachments for a test |
| Get all attachments for a test run |
| Get all attachments for a test plan |
| Get all attachments for a test plan entry |
| Delete an attachment |
Usage Examples
Once configured, you can interact with TestRail through your AI assistant:
Get all projects
"List all TestRail projects"Create a test case
"Create a new test case in section 123 with title 'Verify login functionality'"Add test results
"Add a passed result for test case 456 in run 789 with comment 'All checks passed'"Get test run results
"Show me all results for test run 101"Test Result Status IDs
When adding results, use these standard status IDs:
Status ID | Status |
1 | Passed |
2 | Blocked |
3 | Untested |
4 | Retest |
5 | Failed |
Development
Building from Source
# Clone the repository
git clone https://github.com/TenBarrel6/testrail-mcp.git
cd testrail-mcp
# Install dependencies
npm install
# Build
npm run build
# Run in development mode
npm run devScripts
npm run build- Compile TypeScript to JavaScriptnpm run start- Run the compiled servernpm run dev- Run in development mode with hot reload
Requirements
Node.js 18+
TestRail instance with API access enabled
Valid TestRail API key
Troubleshooting
"Not authenticated" Error
Ensure your environment variables are correctly set:
TESTRAIL_URLshould include the protocol (https://)TESTRAIL_USERNAMEshould be your email addressTESTRAIL_API_KEYshould be a valid API key (not your password)
Connection Issues
Verify your TestRail URL is accessible
Check if API access is enabled in your TestRail administration settings
Ensure your user has appropriate permissions
MCP Client Not Connecting
Restart your MCP client (Cursor/Claude Desktop)
Verify the configuration JSON syntax is valid
Check that
npxis available in your PATH
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
Fork the repository
Create your feature branch (
git checkout -b feature/amazing-feature)Commit your changes (
git commit -m 'Add some amazing feature')Push to the branch (
git push origin feature/amazing-feature)Open a Pull Request
License
This project is licensed under the MIT License - see the LICENSE file for details.
Author
Ruslan Sapun
GitHub: @TenBarrel6
Links
Available Tools
77 toolsadd_attachment_to_caseC
Add an attachment to a test case
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | Case ID | |
| file_path | Yes | Path to the file to upload |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Add an attachment' implies a write/mutation operation, but the description doesn't disclose behavioral traits like required permissions, file size limits, supported file types, whether this overwrites existing attachments, or what happens on success/failure. This is inadequate for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for what it communicates.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns, error conditions, or important behavioral constraints. Given the complexity of file attachment operations and the lack of structured metadata, the description should provide more context about how the operation works.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters (case_id and file_path). The description doesn't add any meaning beyond what the schema provides - it doesn't explain what constitutes a valid case_id or file_path format. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Add an attachment to a test case' clearly states the action (add) and target resource (attachment to test case). It's specific but doesn't distinguish from sibling attachment tools like 'add_attachment_to_plan' or 'add_attachment_to_result' which have similar structures but different targets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives. With multiple attachment-related siblings (add_attachment_to_plan, add_attachment_to_result, etc.), the description doesn't explain that this is specifically for test cases versus other entities. No prerequisites or exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_attachment_to_planC
Add an attachment to a test plan
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| file_path | Yes | Path to the file to upload |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Add an attachment' implies a write/mutation operation, it doesn't specify permissions required, side effects (e.g., file size limits, supported formats), or response behavior (e.g., success/failure indicators). This leaves significant gaps for an agent to understand how to use it safely and effectively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words, directly stating the tool's purpose. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, and the tool being a mutation operation (adding attachments), the description is incomplete. It doesn't address behavioral aspects like error handling, return values, or system constraints, which are crucial for an agent to invoke it correctly in a test management context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear parameter descriptions in the schema ('Plan ID' and 'Path to the file to upload'). The description adds no additional semantic context beyond what the schema provides, such as format examples or constraints, so it meets the baseline for high schema coverage without compensating value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add an attachment') and target resource ('to a test plan'), which is specific and unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_plan_entry', which have similar structures but target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'add_attachment_to_case' or 'add_attachment_to_plan_entry', nor does it mention prerequisites, constraints, or exclusions. It merely states what the tool does without contextual usage information.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_attachment_to_plan_entryC
Add an attachment to a test plan entry
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| entry_id | Yes | Plan entry ID | |
| file_path | Yes | Path to the file to upload |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action is to 'Add an attachment', implying a write/mutation operation, but doesn't specify permissions required, side effects (e.g., if it overwrites existing attachments), rate limits, or error conditions. This leaves significant gaps for an agent to understand the tool's behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It's appropriately sized for a tool with a straightforward action and well-documented schema.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't address behavioral aspects like mutation effects, error handling, or return values, which are critical for a write operation. For a tool with 3 parameters and no structured safety hints, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear parameter descriptions in the schema (plan_id, entry_id, file_path). The description doesn't add any meaning beyond this, such as explaining relationships between parameters (e.g., entry_id must belong to plan_id) or file format constraints. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add an attachment') and target resource ('to a test plan entry'), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_plan', which have similar naming patterns but target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'add_attachment_to_case' or 'add_attachment_to_plan'. It lacks context about prerequisites (e.g., needing an existing plan entry) or exclusions (e.g., not for adding attachments to other entities).
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_attachment_to_resultC
Add an attachment to a test result
| Name | Required | Description | Default |
|---|---|---|---|
| result_id | Yes | Result ID | |
| file_path | Yes | Path to the file to upload |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the action ('Add') but doesn't disclose behavioral traits like required permissions, whether this is a write operation, potential side effects (e.g., file size limits, supported formats), or error handling. This is inadequate for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that this is a mutation tool with no annotations and no output schema, the description is incomplete. It lacks crucial context like behavioral details (e.g., permissions, side effects), usage guidelines relative to siblings, and information about return values or errors, leaving significant gaps for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with clear parameter names and descriptions ('Result ID', 'Path to the file to upload'). The description doesn't add any meaning beyond this, such as format details or constraints, but the schema provides a solid baseline, so a score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add an attachment') and target resource ('to a test result'), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_run', which have similar structures but different targets.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'add_attachment_to_case' or 'add_attachment_to_run'. It lacks context about prerequisites, such as whether the result must exist or be in a specific state, and doesn't mention any exclusions or complementary tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_attachment_to_runC
Add an attachment to a test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| file_path | Yes | Path to the file to upload |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Add' implies a write/mutation operation, the description doesn't address important behavioral aspects like required permissions, file size limits, supported file types, whether this operation is idempotent, or what happens on failure. For a mutation tool with zero annotation coverage, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that states the tool's purpose without unnecessary words. It's appropriately sized for a simple tool and front-loads the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is inadequate. It doesn't explain what happens after the attachment is added, what the response looks like, error conditions, or important constraints. Given the complexity of file upload operations and the lack of structured metadata, the description should provide more contextual information.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters (run_id and file_path) having clear descriptions in the schema. The description doesn't add any additional semantic context beyond what the schema already provides about these parameters, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add an attachment') and target resource ('to a test run'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_plan', which have similar structures but target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With multiple attachment-related tools in the sibling list (add_attachment_to_case, add_attachment_to_plan, add_attachment_to_plan_entry, add_attachment_to_result), there's no indication of when this specific tool is appropriate versus those other attachment tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_caseC
Create a new test case
| Name | Required | Description | Default |
|---|---|---|---|
| section_id | Yes | Section ID | |
| title | Yes | Test case title | |
| template_id | No | Template ID (optional) | |
| type_id | No | Test case type ID (optional) | |
| priority_id | No | Priority ID (optional) | |
| estimate | No | Time estimate (optional) | |
| milestone_id | No | Milestone ID (optional) | |
| refs | No | References/Requirements (optional) | |
| custom_preconds | No | Preconditions (optional) | |
| custom_steps | No | Test steps (optional) | |
| custom_expected | No | Expected result (optional) | |
| custom_autostat | No | Automation Status (optional) | |
| custom_steps_separated | No | Separated test steps (optional). Array of step objects with content/expected or shared_step_id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. 'Create a new test case' implies a write operation but reveals nothing about permissions needed, side effects, error conditions, or what happens on success (e.g., whether a case ID is returned). This leaves significant gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple creation operation and front-loads the essential action without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 13 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what happens after creation (e.g., returns a case ID), error handling, or system constraints. The high parameter count and lack of behavioral context make this description incomplete for effective tool use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 13 parameters. The description adds no parameter-specific information beyond what's in the schema, which meets the baseline for high coverage but doesn't provide additional context like typical values or relationships between parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new test case' clearly states the verb ('Create') and resource ('test case'), making the purpose immediately understandable. However, it doesn't differentiate this tool from other creation tools like 'add_section' or 'add_milestone' beyond the resource type, which prevents a perfect score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools available (e.g., 'update_case', 'delete_case', 'get_cases'), there's no indication of prerequisites, appropriate contexts, or distinctions from related operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_milestoneC
Create a new milestone
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| name | Yes | Milestone name | |
| description | No | Milestone description (optional) | |
| due_on | No | Due date as UNIX timestamp (optional) | |
| parent_id | No | Parent milestone ID for sub-milestones (optional) | |
| refs | No | References (optional) | |
| start_on | No | Start date as UNIX timestamp (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose permissions required, whether creation is idempotent, error handling, or what happens on success (e.g., returns a milestone ID). This is inadequate for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., the new milestone object or ID), error conditions, or dependencies like project existence. Given the complexity of 7 parameters and mutation nature, more context is needed for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents all 7 parameters, including optionality and data types. The description adds no additional meaning beyond implying creation, which is already clear from the tool name. Baseline 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new milestone' clearly states the action (create) and resource (milestone), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'add_case' or 'add_suite' beyond the resource type, missing explicit distinction in scope or hierarchy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'update_milestone' or other 'add_' tools. The description lacks context about prerequisites, such as needing a valid project, or exclusions, leaving the agent to infer usage from the input schema alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_planC
Create a new test plan
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| name | Yes | Plan name | |
| description | No | Plan description (optional) | |
| milestone_id | No | Milestone ID (optional) | |
| entries | No | Array of plan entries/test runs (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. It states 'Create' implying a write operation, but doesn't disclose permissions needed, side effects, error conditions, or what happens on success (e.g., returns a plan ID).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable and appropriately sized for its purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a creation tool with 5 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what a 'test plan' is in this context, what happens after creation, or how it relates to other entities like projects or milestones mentioned in parameters.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are documented in the schema. The description adds no additional parameter context beyond implying creation of a 'test plan', which aligns with the schema but doesn't provide extra semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Create') and resource ('new test plan'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'add_plan_entry' or 'update_plan', but it's not misleading or tautological.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'update_plan' or 'add_plan_entry'. It doesn't mention prerequisites, dependencies, or contextual triggers for creation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_plan_entryC
Add test runs to a test plan
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| suite_id | Yes | Suite ID | |
| name | No | Entry name (optional) | |
| description | No | Entry description (optional) | |
| assignedto_id | No | User ID to assign (optional) | |
| include_all | No | Include all test cases (optional) | |
| case_ids | No | Specific case IDs (optional) | |
| config_ids | No | Configuration IDs (optional) | |
| refs | No | References (optional) | |
| runs | No | Array of test runs with configurations (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. 'Add test runs to a test plan' implies a write operation but doesn't specify permissions needed, whether it's idempotent, what happens on failure, or the format of results. For a mutation tool with 10 parameters, this minimal description fails to provide necessary behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with zero wasted words. It's appropriately sized for a tool name that's self-explanatory ('add_plan_entry'), though this conciseness comes at the cost of completeness. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (10 parameters, mutation operation, no annotations, no output schema), the description is incomplete. It doesn't explain what a 'plan entry' is, how test runs are structured, what happens when adding runs, or what the tool returns. For a tool with this many parameters and no structured guidance, more context is needed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter semantics beyond implying 'test runs' relate to the 'runs' parameter. This meets the baseline of 3 since the schema does the heavy lifting, but the description doesn't enhance understanding of how parameters interact.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Add test runs to a test plan' clearly states the action (add) and target (test runs to a test plan), which is adequate. However, it doesn't differentiate from sibling tools like 'add_run_to_plan_entry' or 'add_run', leaving ambiguity about whether this creates new runs or adds existing ones to a plan. The purpose is clear but lacks sibling distinction.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools like 'add_run', 'add_run_to_plan_entry', and 'add_plan', there's no indication of prerequisites, context, or exclusions. This leaves the agent guessing about appropriate use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_resultC
Add a test result
| Name | Required | Description | Default |
|---|---|---|---|
| test_id | Yes | Test ID | |
| status_id | Yes | Status ID (1=Passed, 5=Failed, etc.) | |
| comment | No | Comment (optional) | |
| elapsed | No | Time elapsed (optional) | |
| defects | No | Defect IDs (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description should disclose behavioral traits like side effects, required permissions, or return values. It only states the basic function without any additional context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is concise and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of output schema and annotations, the description is too minimal for a tool with 5 parameters. It fails to explain the result of the operation or any constraints.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with all parameters described. The description adds no extra meaning beyond the schema, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Add' and the resource 'a test result', which is distinct from siblings like 'add_case' or 'add_result_for_case'. However, it could be more specific by mentioning it's for a test case result.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives such as 'add_result_for_case' or 'add_results_for_cases'. The description lacks context for proper selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_result_for_caseB
Add a test result for a specific case in a run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| case_id | Yes | Case ID | |
| status_id | Yes | Status ID | |
| comment | No | Comment (optional) | |
| elapsed | No | Time elapsed (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description only states the action. It does not disclose behavioral traits like whether it creates or updates an existing result, required permissions, or side effects. For a write operation, this is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that front-loads the key information. Every word is necessary and contributes to understanding the tool's purpose. No redundant information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description lacks details about the tool's output (no output schema), edge cases (e.g., duplicate results), prerequisites, or how parameters like 'elapsed' are formatted. Given 5 parameters and no annotations, it is minimally complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides for each parameter. It does not clarify formats or valid values (e.g., for 'elapsed' or 'status_id').
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add'), the resource ('test result'), and the context ('for a specific case in a run'). It effectively distinguishes from sibling tools like 'add_result' and 'add_results_for_cases'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for adding a result to a specific case in a run, but does not provide explicit guidance on when to use it over alternatives (e.g., 'add_result' or 'add_results_for_cases'). No when-not-to-use or prerequisites are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_resultsC
Add multiple test results by test IDs
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| results | Yes | Array of results |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Add' implies a write operation, but the description doesn't specify whether this requires authentication, what happens on success or error (e.g., partial updates), or if there are rate limits. It also doesn't clarify if 'add' means create new entries or update existing ones, leaving behavioral traits ambiguous. This is a significant gap for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and resource, making it easy to parse. However, it could be slightly more informative by clarifying the scope of 'add' (e.g., 'create' or 'append'), but as-is, it avoids redundancy and waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a bulk write operation with no annotations and no output schema, the description is inadequate. It doesn't address key aspects like authentication needs, error handling, or what the tool returns (e.g., success status, IDs of added results). For a tool that modifies data and has many similar siblings, more context is needed to ensure correct usage and avoid confusion.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear documentation for 'run_id' and 'results' (including nested properties like test_id, status_id). The description adds minimal value beyond the schema, mentioning 'test IDs' which aligns with the 'test_id' parameter but doesn't explain the relationship between test IDs and results or provide additional context like format examples. With high schema coverage, the baseline score of 3 is appropriate as the schema does most of the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Add multiple test results by test IDs' clearly states the action (add) and resource (test results), but it's somewhat vague about what 'add' means in this context. It doesn't specify whether this creates new results, appends to existing ones, or updates them. Compared to sibling tools like 'add_result' (singular) and 'add_results_for_cases', it distinguishes itself by focusing on test IDs rather than cases, but the distinction isn't explicitly articulated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'add_result' (singular), 'add_results_for_cases', or 'add_result_for_case'. It doesn't mention prerequisites, such as needing an existing run or test, or specify scenarios where bulk addition by test IDs is preferred over other methods. Without this context, an agent might struggle to choose the correct tool among similar siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_results_for_casesB
Add multiple test results for cases in a run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| results | Yes | Array of results |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description only states the action ('add') without disclosing side effects, permissions needed, or behavior on errors (e.g., if run_id is invalid). The burden is on the description, which is insufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence with no unnecessary words. It conveys the essential purpose efficiently, though it could benefit from a bit more detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has nested parameters without full documentation (e.g., what status_id values are valid) and no output schema. The description does not explain the structure of the results array or provide sufficient context for correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with basic descriptions ('Run ID', 'Array of results'). The description does not add further meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Add' and the resource 'multiple test results for cases in a run', distinguishing it from sibling tools like add_result (single) and add_result_for_case (single for case).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. Sibling tools exist for adding single results, but the description does not explicitly differentiate or mention when to prefer this bulk operation.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_runB
Create a new test run
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| suite_id | No | Suite ID (optional) | |
| name | Yes | Run name | |
| description | No | Run description (optional) | |
| milestone_id | No | Milestone ID (optional) | |
| assignedto_id | No | User ID to assign (optional) | |
| include_all | No | Include all test cases (optional) | |
| case_ids | No | Specific case IDs (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description only says 'Create a new test run', which implies mutation, but lacks detail on side effects, authorization needed, or what happens upon success. Without annotations, the description should compensate but does not.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, but it is too brief. It wastes no words but also provides minimal information. A single sentence suffices for the purpose, but it could be more informative without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 8 parameters, no output schema, and no annotations, the description is insufficient. It does not explain return values, error states, or how parameters like include_all and case_ids interact. The agent may lack context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for its 8 parameters, so the description adds no additional meaning. The baseline of 3 is appropriate as the schema already documents parameter semantics adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Create' and resource 'new test run', which is specific and unambiguous. It succinctly indicates the tool's function without ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool vs alternatives like update_run or add_case. There is no mention of prerequisites or context that would help the agent decide appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_run_to_plan_entryC
Add a test run to an existing plan entry
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| entry_id | Yes | Plan entry ID | |
| config_ids | Yes | Configuration IDs | |
| description | No | Run description (optional) | |
| assignedto_id | No | User ID to assign (optional) | |
| include_all | No | Include all test cases (optional) | |
| case_ids | No | Specific case IDs (optional) | |
| refs | No | References (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. While 'add' implies a write/mutation operation, the description doesn't specify permissions required, whether this creates a new run or associates existing runs, what happens on success/failure, rate limits, or side effects. For a tool that modifies data with 8 parameters, this is insufficient behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence that efficiently communicates the core function. There's no wasted verbiage, repetition, or unnecessary elaboration. It's appropriately sized for the tool's complexity and gets straight to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 8 parameters, no annotations, and no output schema, the description is incomplete. It doesn't address what the tool returns, error conditions, authentication requirements, or how it differs from similar tools. The description provides only the most basic functional statement without the context needed for effective use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all 8 parameters with basic descriptions. The tool description adds no additional parameter information beyond what's in the schema - it doesn't explain relationships between parameters (like how 'include_all' interacts with 'case_ids'), parameter constraints, or practical usage examples. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('add a test run') and target resource ('to an existing plan entry'), providing specific verb+resource pairing. However, it doesn't differentiate from sibling tools like 'add_run' or 'add_run_to_plan_entry' vs 'update_run_in_plan_entry', which would require more specificity about what distinguishes this particular addition operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools like 'add_run', 'add_plan_entry', 'update_run_in_plan_entry', and 'delete_run_from_plan_entry', there's no indication of prerequisites, appropriate contexts, or when other tools might be more suitable. The description simply states what it does without usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_sectionC
Create a new section
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| suite_id | No | Suite ID (optional) | |
| parent_id | No | Parent section ID (optional) | |
| name | Yes | Section name | |
| description | No | Section description (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'Create a new section' without mentioning effects, permissions, idempotency, or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, unambiguous sentence. It is as concise as possible with no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, the description should mention that the created section object is returned. It also lacks context on how this tool fits among siblings (e.g., when to use add_section vs add_suite). The description is inadequate for an agent to invoke correctly without prior knowledge.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, so each parameter is documented. The tool description adds no extra meaning beyond the schema, such as explaining the relationship between suite_id and parent_id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new section' clearly states the verb (create) and resource (section). It distinguishes from siblings like add_case or add_suite by specifying the resource type, but does not elaborate on scope or nuances.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives such as add_suite or update_section. No context about prerequisites or conditions is given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
add_suiteB
Create a new test suite
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| name | Yes | Suite name | |
| description | No | Suite description (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, yet the description does not disclose any behavioral traits beyond the basic action, such as required project existence or uniqueness constraints.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single efficient sentence, but it could include more context without becoming verbose, slightly limiting its usefulness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Without an output schema or annotations, the description fails to explain return values or usage context, leaving a significant gap in completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the tool description adds no additional meaning beyond what the schema already provides, warranting a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Create a new test suite' clearly states the specific verb and resource, distinguishing it from sibling tools like add_case or add_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for creating suites but provides no explicit guidance on when to use it versus alternatives, nor any exclusions or prerequisites.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
close_planC
Close a test plan
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. 'Close a test plan' implies a mutation operation, but it doesn't specify whether this is reversible, what permissions are required, if it affects associated runs/cases, or what the outcome looks like. For a destructive-sounding tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately understandable. For such a simple tool, this brevity is appropriate and effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (a mutation operation with potential side effects), lack of annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like idempotency, error conditions, or return values. For a 'close' operation in a test management context, more context is needed to use it safely and effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage (the 'plan_id' parameter is documented as 'Plan ID'), so the baseline is 3. The description adds no additional parameter context beyond what the schema provides—it doesn't explain where to find the plan_id, format constraints, or validation rules. This meets minimum viability but offers no extra semantic value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Close a test plan' clearly states the verb ('close') and resource ('test plan'), providing basic purpose. However, it doesn't differentiate from sibling tools like 'close_run' or explain what 'closing' means in this context (e.g., marking as completed vs. archiving). The purpose is understandable but lacks specificity compared to alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether the plan must be open), exclusions (e.g., cannot close if active runs exist), or related tools like 'update_plan' for status changes. With many sibling tools for managing plans, this omission leaves the agent guessing about appropriate usage contexts.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
close_runB
Close a test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description does not disclose what closing a run entails (e.g., archiving, preventing further results, required permissions).
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no extraneous information, perfectly concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema and no annotations, the description fails to explain return values, side effects, or error states (e.g., closing an already closed run). Incomplete for a mutation tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the parameter description ('Run ID') is clear. The tool description adds no extra meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('close') and the resource ('test run'), distinguishing it from sibling tools like add_run, update_run, get_run.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives, such as update_run or deleting a run. Lacks context about prerequisites or post-conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
copy_cases_to_sectionC
Copy test cases to another section
| Name | Required | Description | Default |
|---|---|---|---|
| section_id | Yes | Target section ID | |
| case_ids | Yes | Array of case IDs to copy |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'copy' but does not clarify if this creates duplicates, retains originals, requires permissions, or handles errors. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its effects and requirements.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste—'Copy test cases to another section'—front-loading the core action. It is appropriately sized for the tool's complexity, with no redundant or verbose elements.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool involves mutation (copying cases) with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits (e.g., duplication effects, permissions), error handling, or return values. For a tool with two required parameters and potential side effects, more context is needed to ensure safe and correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with clear descriptions for 'section_id' (target section ID) and 'case_ids' (array of case IDs to copy). The description adds no additional semantic context beyond what the schema provides, such as format details or constraints. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'copy' and the resource 'test cases', specifying the destination 'to another section'. It distinguishes from the sibling 'move_cases_to_section' by implying copying rather than moving, though this distinction could be more explicit. The purpose is specific and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'move_cases_to_section' or 'add_case'. It lacks context about prerequisites, such as whether the source cases or target section must exist, or any constraints on copying. Usage is implied only by the action described.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_attachmentC
Delete an attachment
| Name | Required | Description | Default |
|---|---|---|---|
| attachment_id | Yes | Attachment ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Delete' implies a destructive mutation, but the description doesn't specify whether this action is reversible, requires specific permissions, has side effects (e.g., on linked cases or runs), or provides confirmation. This is a significant gap for a mutation tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with zero wasted words, making it highly concise and front-loaded. It efficiently communicates the core action without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks critical context like behavioral traits (e.g., irreversibility, permissions), usage guidelines, or output expectations. Given the complexity and risk of deletion, this minimal description is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with the single parameter 'attachment_id' documented as 'Attachment ID'. The description doesn't add any meaning beyond this, such as format examples or sourcing guidance. With high schema coverage, the baseline score of 3 is appropriate as the schema handles the parameter documentation adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete an attachment' clearly states the action (delete) and the resource (attachment), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'delete_case' or 'delete_plan', which follow the same pattern but target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an attachment ID), exclusions, or relationships with sibling tools like 'get_attachment' for verification or attachment-specific tools for context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_caseB
Delete a test case
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | Case ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and the description only states 'delete' without detailing whether deletion is permanent, cascading, or requires authentication.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is concise and to the point, but could include a bit more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple delete tool with one parameter, the description is minimally complete but lacks details on irreversibility or effects on related data.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with parameter description 'Case ID', which already conveys the meaning. The description adds no extra insight beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a test case' clearly states the action and resource, distinguishing it from sibling tools like add_case and update_case.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., update_case to deactivate instead). Lacks when-not-to-use or prerequisite conditions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_casesC
Delete multiple test cases
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| suite_id | No | Suite ID (required for multi-suite projects) | |
| case_ids | Yes | Array of case IDs to delete | |
| soft | No | Set to 1 to preview deletion without executing (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the action ('delete') without disclosing critical behavioral traits. It doesn't mention whether deletion is permanent or reversible, what permissions are required, whether it affects related data (like attachments or results), or what happens on success/failure. For a destructive operation, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient phrase with zero wasted words. It's front-loaded with the core action and resource, making it immediately understandable despite its brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It lacks behavioral context (permissions, reversibility, side effects), usage differentiation from siblings, and any information about return values or error conditions. The schema covers parameters well, but the description doesn't compensate for other gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional semantic context about parameters beyond implying 'multiple' cases via 'case_ids'. It doesn't explain relationships between parameters (e.g., when 'suite_id' is needed) or provide examples. Baseline 3 is appropriate given the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete multiple test cases' clearly states the verb (delete) and resource (test cases), and specifies 'multiple' to indicate batch operation. However, it doesn't differentiate from sibling 'delete_case' which presumably deletes single cases, leaving some ambiguity about when to choose one over the other.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives like 'delete_case' (for single deletions) or other deletion tools in the sibling list. The description mentions 'multiple' but doesn't explicitly contrast with single-case deletion or provide context about prerequisites or constraints.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_milestoneC
Delete a milestone
| Name | Required | Description | Default |
|---|---|---|---|
| milestone_id | Yes | Milestone ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states 'Delete a milestone,' implying a destructive mutation, but doesn't disclose behavioral traits such as permissions required, whether deletion is permanent or reversible, error handling, or side effects. This is a significant gap for a destructive tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise ('Delete a milestone') and front-loaded with the core action. It wastes no words, making it efficient for quick understanding, though this brevity contributes to gaps in other dimensions.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's destructive nature, no annotations, and no output schema, the description is incomplete. It fails to address critical context like what happens post-deletion, success/error responses, or dependencies, making it inadequate for safe and effective use by an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter (milestone_id), so the schema already documents it adequately. The description doesn't add parameter details, but with 0 parameters needing extra explanation, a baseline of 4 is appropriate as it doesn't detract from clarity.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a milestone' clearly states the action (delete) and resource (milestone), which is adequate. However, it doesn't differentiate this tool from other deletion tools in the sibling list (e.g., delete_case, delete_plan), making it somewhat vague in context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description lacks context about prerequisites (e.g., milestone existence), exclusions, or comparisons to related tools like update_milestone or get_milestone, leaving usage unclear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_planC
Delete a test plan
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. While 'Delete' clearly indicates a destructive operation, the description doesn't address critical behavioral aspects: whether deletion is permanent or reversible, what permissions are required, whether associated data (like plan entries or attachments) are also deleted, or what the response looks like. For a destructive tool with zero annotation coverage, this represents substantial missing context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is maximally concise at three words, with zero wasted language. It's appropriately sized for a simple destructive operation and gets straight to the point without unnecessary elaboration. The brevity is appropriate given the tool's straightforward purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive operation with no annotations and no output schema, the description is insufficiently complete. It doesn't address the behavioral implications of deletion, provide usage guidance relative to sibling tools, or explain what happens after invocation. Given the tool's potential impact and the rich ecosystem of related tools, more contextual information would help an AI agent use it appropriately and safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'plan_id' documented as 'Plan ID'. The description doesn't add any meaningful semantic context beyond what the schema already provides - it doesn't explain what constitutes a valid plan ID, where to find it, or format requirements. With complete schema coverage, the baseline score of 3 is appropriate as the schema does the parameter documentation work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a test plan' clearly states the verb ('Delete') and resource ('test plan'), making the basic purpose understandable. However, it doesn't differentiate this from sibling tools like 'delete_case', 'delete_milestone', or 'delete_plan_entry' - all of which also delete specific resources in what appears to be a test management system. The description is functional but generic within this context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance about when to use this tool versus alternatives. There's no mention of prerequisites (e.g., whether the plan must be closed first), consequences (e.g., what happens to associated test cases or runs), or when not to use it. In a system with multiple deletion tools, this lack of differentiation is a significant gap for an AI agent trying to select the right tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_plan_entryC
Delete a test plan entry
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| entry_id | Yes | Plan entry ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Delete' implies a destructive mutation, but the description doesn't disclose whether this requires special permissions, whether deletion is permanent or reversible, what happens to associated data, or any rate limits. For a destructive operation with zero annotation coverage, this is a significant behavioral transparency gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a simple deletion operation and front-loads the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive mutation tool with no annotations and no output schema, the description is inadequate. It doesn't address behavioral aspects like permissions, permanence, side effects, or error conditions. Given the complexity of a deletion operation in what appears to be a test management system, more contextual information would be expected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both parameters clearly documented in the schema. The description adds no additional parameter information beyond what's already in the structured schema. With complete schema coverage, the baseline score of 3 is appropriate as the description doesn't need to compensate for schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the target resource ('a test plan entry'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'delete_plan' or 'delete_case', but the verb+resource combination is specific enough for basic understanding.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling deletion tools (delete_plan, delete_case, delete_run, etc.), there's no indication of what distinguishes deleting a plan entry from deleting other entities or when this specific deletion operation is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_runC
Delete a test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| soft | No | Set to 1 to preview deletion without executing (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Delete' implies a destructive mutation, but the description doesn't disclose whether deletion is permanent, reversible, requires specific permissions, or has side effects (e.g., cascading deletions). The optional 'soft' parameter hints at preview capability, but this isn't explained in the description itself.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable. Every word earns its place in conveying the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool with no annotations and no output schema, the description is inadequate. It doesn't explain what 'delete' entails (permanent vs. soft delete), what happens to associated data, error conditions, or return values. The agent lacks critical context needed to use this tool safely and effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no additional meaning about 'run_id' or 'soft' parameters beyond what's in the schema. This meets the baseline score of 3 when schema coverage is high, but doesn't compensate with extra context.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a test run' clearly states the verb ('Delete') and resource ('a test run'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'delete_case', 'delete_plan', or 'delete_run_from_plan_entry', which all follow similar patterns but target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling deletion tools (e.g., delete_case, delete_plan, delete_run_from_plan_entry), there's no indication of which resource type this applies to or any prerequisites for deletion. The agent must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_run_from_plan_entryC
Delete a test run from a plan entry
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the destructive action ('Delete') but doesn't mention permissions required, whether the deletion is permanent or reversible, or any side effects (e.g., impact on related data). This leaves significant gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with no wasted words. It's front-loaded with the core action and target, making it highly efficient and easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive tool with no annotations and no output schema, the description is insufficient. It lacks details on behavioral implications, error conditions, or what happens post-deletion. Given the complexity of mutation operations, more context is needed to ensure safe and correct usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description doesn't add any parameter details beyond the input schema, which has 100% coverage for the single parameter 'run_id'. Since the schema fully documents the parameter, the baseline score of 3 is appropriate, as no additional semantic value is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Delete') and the target ('a test run from a plan entry'), making the purpose immediately understandable. It distinguishes from sibling tools like 'delete_run' by specifying the context of a plan entry, though it doesn't explicitly contrast with all deletion alternatives.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'delete_run' or 'delete_plan_entry'. The description assumes the context is known but doesn't specify prerequisites, conditions, or exclusions for usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_sectionC
Delete a section
| Name | Required | Description | Default |
|---|---|---|---|
| section_id | Yes | Section ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must convey behavioral traits. It only states the action (delete) without disclosing consequences (e.g., cascading deletes, permanence, required permissions). This is minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the purpose. It is appropriately sized, though it could include more detail without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description is incomplete. A deletion tool should specify what happens to associated data, safety considerations, or return behavior to be fully informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% for the single parameter (section_id). The description adds no additional meaning beyond the schema's 'Section ID', so the value is at baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Delete a section' clearly states the verb (delete) and resource (section), making the purpose unambiguous. However, it does not differentiate from sibling tools like delete_case, but the name itself aids differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. There is no indication of when to use this tool versus alternatives (e.g., update_section) or any prerequisites or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_attachmentC
Get/download an attachment by ID
| Name | Required | Description | Default |
|---|---|---|---|
| attachment_id | Yes | Attachment ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'Get/download' which implies a read operation, but doesn't specify whether this requires authentication, what format the download returns (e.g., file content vs. metadata), or any rate limits. The description is minimal and lacks important behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise - a single phrase that communicates the core purpose without any wasted words. It's appropriately sized for a simple retrieval tool with one parameter.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description is inadequate. It doesn't explain what format the attachment is returned in (e.g., binary data, download link, metadata), whether authentication is required, or how this differs from similar 'get_attachments_for_*' tools. The minimal description leaves too many questions unanswered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents the single 'attachment_id' parameter. The description adds no additional semantic context about the parameter beyond what's in the schema (e.g., where to find attachment IDs, format requirements). Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get/download') and resource ('an attachment by ID'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_attachments_for_case' or 'delete_attachment' beyond the basic operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get_attachments_for_case' or 'delete_attachment'. The description only states what it does, not when it's appropriate or what prerequisites might exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_attachments_for_caseC
Get all attachments for a test case
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | Case ID | |
| limit | No | Limit results (optional) | |
| offset | No | Offset for pagination (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It implies a read operation ('Get'), but doesn't disclose behavioral traits such as permissions needed, rate limits, pagination behavior (beyond schema hints), or what 'attachments' include (e.g., file types, sizes). This is a significant gap for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words, making it easy to parse. However, it's front-loaded but could be more structured (e.g., by explicitly noting it's a list operation).
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'attachments' entail (e.g., metadata, files), how results are returned, or error conditions. For a tool with 3 parameters and list functionality, more context is needed to guide the agent effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema documents all parameters (case_id, limit, offset). The description adds no meaning beyond the schema—it doesn't explain parameter relationships (e.g., limit/offset for pagination) or constraints. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the purpose ('Get all attachments for a test case') with a clear verb ('Get') and resource ('attachments'), but it's vague about scope ('all' is ambiguous without context) and doesn't distinguish from sibling tools like 'get_attachment' (singular) or 'get_attachments_for_plan' (different resource). It's adequate but lacks specificity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives is provided. The description doesn't mention sibling tools like 'get_attachment' (for a single attachment) or 'get_attachments_for_plan' (for attachments on plans), leaving the agent to infer usage from context alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_attachments_for_planC
Get all attachments for a test plan
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| limit | No | Limit results (optional) | |
| offset | No | Offset for pagination (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the basic action. It doesn't disclose behavioral traits such as whether this is a read-only operation, potential rate limits, authentication needs, pagination behavior (implied by parameters but not explained), or what happens if the plan_id is invalid. This leaves significant gaps for a tool with parameters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for its simple function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 3 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain return values (e.g., format of attachments list), error conditions, or behavioral context, leaving the agent with insufficient information for reliable use beyond basic parameter mapping.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so the schema already documents all parameters (plan_id, limit, offset) adequately. The description adds no additional meaning beyond implying 'all attachments' relates to plan_id, which is already clear from the schema. This meets the baseline of 3 for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'all attachments for a test plan', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_attachments_for_case' or 'get_attachments_for_plan_entry', which would require a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_attachment' (for a single attachment) or other attachment-fetching siblings. It lacks any context about prerequisites, exclusions, or comparative use cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_attachments_for_plan_entryC
Get all attachments for a test plan entry
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| entry_id | Yes | Plan entry ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Get') but doesn't clarify if this is a read-only operation, what permissions are required, whether it returns paginated results, or the format of the returned attachments. For a retrieval tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place without redundancy or unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of a retrieval tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'attachments' entail (e.g., files, links), the return format, or error handling. For a tool that likely returns structured data, more context is needed to guide the agent effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with clear descriptions for both parameters (plan_id as a number, entry_id as a string). The description adds no additional meaning beyond the schema, such as explaining what a 'plan entry' is or providing examples. Given the high schema coverage, the baseline score of 3 is appropriate, as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('all attachments for a test plan entry'), making the purpose specific and understandable. However, it doesn't explicitly distinguish this tool from sibling tools like 'get_attachments_for_case' or 'get_attachments_for_plan', which have similar naming patterns but target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid plan_id and entry_id), exclusions, or comparisons to sibling tools like 'get_attachment' (singular) or other attachment retrieval tools, leaving the agent to infer usage from naming alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_attachments_for_runB
Get all attachments for a test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| limit | No | Limit results (optional) | |
| offset | No | Offset for pagination (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Get all attachments' but doesn't clarify if this is a read-only operation, what permissions are required, whether it's paginated (implied by limit/offset parameters but not stated), or what the return format looks like. For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a straightforward retrieval tool, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity (retrieval with optional pagination), no annotations, and no output schema, the description is minimally adequate. It states what the tool does but lacks details on behavior, output format, or usage context. For a tool with 3 parameters and no structured safety hints, this leaves the agent with incomplete information to operate it effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents the three parameters (run_id, limit, offset). The description adds no additional meaning beyond the schema's parameter descriptions, such as explaining what 'attachments' include or how pagination works. With high schema coverage, the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all attachments for a test run' clearly states the verb ('Get') and resource ('attachments for a test run'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_attachments_for_case' or 'get_attachments_for_plan', which have similar patterns but target different resources.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid run_id), exclusions, or comparisons to siblings like 'get_attachment' (singular) or 'get_attachments_for_test', leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_attachments_for_testC
Get all attachments for a test
| Name | Required | Description | Default |
|---|---|---|---|
| test_id | Yes | Test ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the action ('Get') but lacks behavioral details: it doesn't specify if this is a read-only operation, what permissions are required, how results are returned (e.g., pagination, format), or error conditions. This is a significant gap for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It is front-loaded and appropriately sized for a simple tool, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It lacks behavioral context (e.g., safety, permissions, return format) and doesn't compensate for the absence of structured data, making it inadequate for a tool that likely returns multiple attachments.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the single parameter 'test_id' documented in the schema as 'Test ID'. The description adds no additional parameter semantics beyond implying the tool uses a test ID, so it meets the baseline of 3 where the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all attachments for a test' clearly states the verb ('Get') and resource ('attachments for a test'), making the purpose unambiguous. It distinguishes from siblings like 'get_attachment' (singular) and 'get_attachments_for_case' (different resource), though it doesn't explicitly contrast them.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. While the description implies it's for retrieving attachments specifically for tests, it doesn't mention prerequisites (e.g., needing a valid test ID), exclusions, or compare it to similar tools like 'get_attachments_for_case' or 'get_attachment'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_caseB
Get a specific test case by ID
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | Case ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as error behavior for non-existent IDs, authentication requirements, or whether the operation is read-only. Minimal information.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise (one sentence) but lacks structure; it does not include any line breaks or additional formatting. It is under-specified for a complete tool description.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has one parameter, no output schema, and no annotations, the description is minimal but technically sufficient for a simple get-by-ID operation. However, it could be improved by noting return type or error handling.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already has 100% coverage by describing the case_id parameter. The description adds no further semantic meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action 'Get' and the resource 'a specific test case by ID', which is precise and distinguishes from sibling tools like get_cases (plural) or get_case_fields.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives (e.g., get_cases for listing). The description does not mention that a valid case_id is required or that the tool is only for individual retrieval.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_case_fieldsA
Get all available test case fields
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full responsibility for conveying behavior. It says 'Get all available test case fields' but does not explain what 'fields' means (e.g., schema definitions, values) or how the result is structured. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no wasted words. It is optimally concise for the simplicity of the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no parameters and no output schema, the description is adequate but minimal. It explains the purpose but does not describe the output format or any prerequisites. Could be more complete given the lack of annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema description coverage is 100% trivially. The description adds no parameter info but does not need to. Score is baseline 4 for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('all available test case fields'). It distinguishes this tool from siblings like get_case or get_cases, which retrieve case data rather than field definitions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like get_case_types or get_sections. With many sibling tools, explicit usage context is missing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_casesB
Get test cases for a project/suite
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| suite_id | No | Suite ID (optional) | |
| section_id | No | Section ID (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided; description does not disclose read-only nature, error handling, or performance characteristics. Minimal behavioral context beyond implied retrieval.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single short sentence conveying core purpose. Could be slightly more structured but remains efficient with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Incomplete given no output schema and 3 parameters. Does not describe return format, pagination, or potential errors. Lacks details needed for robust usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with descriptions for all parameters. Description adds little beyond schema, but at baseline level as it does not contradict or enhance.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states it retrieves test cases for a project/suite, using specific verb and resource. Differentiates from siblings like 'get_case' (singular) and other retrieval tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like 'get_case' or 'get_results'. Lacks context for filtering or when to specify suite_id/section_id.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_case_statusesB
Get all available test case statuses (Enterprise)
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Get') but doesn't describe the return format (e.g., list of status objects, JSON structure), pagination behavior, error handling, or any rate limits. For a read operation with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It front-loads the key information ('Get all available test case statuses') and includes a brief qualifier '(Enterprise)' for added context, making it easy to scan and understand quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but has clear gaps. It lacks details on the return format, which is critical since there's no output schema, and doesn't address behavioral aspects like error handling. For a read-only tool with no structured output documentation, more completeness would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so the schema fully documents that no inputs are required. The description doesn't add parameter details, which is appropriate here, but it could have mentioned if any implicit parameters (like authentication context) are needed. Baseline is 4 for 0 parameters, as the description doesn't need to compensate for gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('all available test case statuses'), with the parenthetical '(Enterprise)' providing additional context about the scope or version. However, it doesn't explicitly differentiate from the sibling tool 'get_statuses' (which appears to be a similar but potentially different tool), leaving some ambiguity about when to use one versus the other.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_statuses' or other status-related tools. It lacks context about prerequisites, such as whether it requires authentication or specific project access, and doesn't mention any exclusions or limitations in usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_case_typesB
Get all available test case types
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so the description carries full responsibility. It does not disclose any behavioral traits such as idempotency, authentication requirements, or effects. The description is minimal and provides no insight into side effects or state.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very short (one sentence) with no wasted words, appropriate for a zero-parameter tool. However, it could be slightly more informative without compromising conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no parameters and no output schema, the description is functionally adequate but lacks context about what the returned data represents (e.g., IDs, names). It does not mention any prerequisites or related operations, leaving room for ambiguity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (no parameters), so the description adds no additional meaning beyond the schema. Baseline is 3, and the description simply restates the resource name without enriching parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and the resource ('all available test case types'). It distinguishes itself from siblings like get_case_fields or get_cases by specifying 'case types' as a distinct entity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives, but the action is straightforward (listing types) and likely used before creation operations. The description implies usage context but lacks detailed when-to-use instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_configsC
Get all configurations for a project
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states a read operation ('Get'), implying it is likely safe and non-destructive, but does not specify details like authentication needs, rate limits, error handling, or the format/scope of returned configurations (e.g., whether it returns all configurations at once or supports pagination). This leaves significant gaps for a tool with no annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, direct sentence with no wasted words, making it easy to parse and front-loaded with the core purpose. It efficiently conveys the essential information without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It does not address behavioral aspects like response format, error conditions, or system constraints, which are critical for a read operation tool. While concise, it fails to provide sufficient context for reliable agent use beyond the basic purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with the single parameter 'project_id' clearly documented. The description adds no additional semantic context beyond implying the parameter is used to scope configurations to a specific project, which is already evident from the schema. This meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('configurations for a project'), making the purpose immediately understandable. However, it does not distinguish this tool from its many sibling 'get_' tools (e.g., get_case, get_plan, get_results), which all follow a similar pattern of retrieving specific data, so it lacks explicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With many sibling tools available (e.g., get_project, get_cases), there is no indication of context, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_current_userB
Get the current authenticated user
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, the description doesn't specify what information is returned, authentication requirements, error conditions, or rate limits. It mentions 'authenticated user' but doesn't clarify what happens if no user is authenticated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that communicates the core purpose without any wasted words. It's appropriately sized for a simple tool with no parameters and gets straight to the point.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a user retrieval tool with no annotations and no output schema, the description is insufficient. It doesn't explain what user information is returned (e.g., ID, name, email, permissions), authentication requirements, or error handling. The agent would need to guess about the response format and behavioral characteristics.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0 parameters and 100% schema description coverage, the baseline is 4. The description appropriately doesn't discuss parameters since none exist, and the schema already fully documents this. No additional parameter information is needed or provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('current authenticated user'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'get_user' or 'get_user_by_email', but the focus on 'current authenticated' user provides some implicit differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'get_user' or 'get_user_by_email'. It doesn't mention prerequisites (e.g., authentication status) or specific contexts where this tool is appropriate versus other user retrieval methods.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_history_for_caseC
Get the edit history for a test case
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | Case ID | |
| limit | No | Limit results (optional) | |
| offset | No | Offset for pagination (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states it 'gets' data (implying read-only), but doesn't disclose behavioral traits like authentication requirements, rate limits, error conditions, or what 'edit history' includes (e.g., timestamps, users, changes). For a tool with no annotation coverage, this is insufficient transparency about how it behaves.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, clear sentence with zero waste. It's appropriately sized for a simple retrieval tool and front-loads the core purpose. Every word earns its place without redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'edit history' returns (e.g., list of changes, timestamps, users), pagination behavior (implied by offset/limit but not described), or error handling. For a tool with 3 parameters and rich sibling context, more completeness is needed to guide the agent effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no additional meaning about parameters beyond implying 'case_id' identifies the test case. With high schema coverage, the baseline is 3 even without parameter details in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('edit history for a test case'), making the purpose understandable. It doesn't explicitly differentiate from siblings like 'get_case' or 'get_results_for_case', but the focus on 'edit history' provides reasonable distinction. A 5 would require explicit sibling comparison.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided about when to use this tool versus alternatives. With many sibling tools for retrieving case-related data (e.g., 'get_case', 'get_results_for_case', 'get_attachments_for_case'), the description offers no context about when edit history is needed versus other case information. This leaves the agent without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_milestoneA
Get a specific milestone by ID
| Name | Required | Description | Default |
|---|---|---|---|
| milestone_id | Yes | Milestone ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description indicates a read operation but does not disclose error handling, permissions, or what happens if the milestone is not found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no wasted words, easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID tool with one parameter and no output schema, the description is mostly complete but could mention the return format.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already explains the parameter. Description adds no extra meaning beyond 'Milestone ID'.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states the verb 'Get' and resource 'a specific milestone by ID', distinguishing it from the sibling tool 'get_milestones'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'get_milestones'. Implied by naming but not stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_milestonesB
Get milestones for a project
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fails to disclose behavioral details such as whether the operation is read-only, idempotent, or requires specific permissions. It does not describe the return type or any side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely brief but lacks structure or elaboration. While concise, it borders on under-specification.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, no output schema, no annotations), the description minimally covers the purpose. However, it could be more complete by indicating that it returns a list of milestones or the expected output.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the single parameter 'project_id' described as 'Project ID'. The tool description adds no extra semantic value beyond the schema, fitting the baseline of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses the specific verb 'Get' and clearly identifies the resource 'milestones for a project'. It distinguishes itself from the sibling 'get_milestone' which retrieves a single milestone.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get_milestone'. No prerequisites or context for usage are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_planC
Get a specific test plan by ID
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. The description only states 'Get a specific test plan by ID', implying a read-only operation, but does not disclose behavior on missing IDs, error handling, or any side effects. The description lacks needed transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise, consisting of a single phrase. It contains no fluff but is perhaps too brief; it could benefit from additional context without sacrificing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, no output schema, and the sibling toolset, the description is minimally adequate. It covers the core action but omits return value context or any edge cases, making it moderately complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter plan_id, which is described as 'Plan ID'. The tool description adds 'by ID' reinforcing the parameter purpose but does not add new meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a specific test plan by ID' clearly states the verb and resource, distinguishing from sibling tools like get_plans which lists plans. However, it does not explicitly differentiate from similar tools like get_case or get_suite, but the resource is specific enough.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get_plans for listing. The description implies usage for a single plan but does not mention prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_plansC
Get test plans for a project
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states a generic read operation ('Get') without mentioning any side effects, rate limits, pagination, or filtering behavior. The input schema covers project_id, but the description omits any additional behavioral traits.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short (one sentence) and front-loaded, but this brevity sacrifices informativeness. While it avoids wasted words, it does not earn its place by providing sufficient detail beyond the tool's name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the absence of an output schema, the description should clarify what is returned (e.g., a list of test plans, details). It does not. The tool has 1 parameter and is simple, but the description fails to cover the return type or any edge cases, leaving the agent with incomplete context.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% for the single parameter 'project_id'. The description does not add any meaning beyond the schema's 'Project ID' label. The baseline score of 3 is appropriate since the schema already documents the parameter adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('test plans') with scope ('for a project'). It is not a tautology and distinguishes from similar verb-noun combinations. However, it does not differentiate from the sibling tool 'get_plan' (singular), which could confuse an agent deciding between the two.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get_plan', 'get_cases', or 'get_sections'. The agent receives no context about the expected output or prerequisites, leaving it to infer usage solely from the name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_prioritiesB
Get all available test case priorities
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states 'Get all available test case priorities', implying a read-only operation, but doesn't disclose behavioral traits like whether it requires authentication, returns a list or object, has pagination, or potential errors. This is inadequate for a tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given 0 parameters, 100% schema coverage, and no output schema, the description is minimally complete but lacks depth. It doesn't explain return values (e.g., format, structure) or behavioral context, which is a gap since annotations are absent. For a simple read tool, it's adequate but could be more informative.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, so no parameter information is needed. The description doesn't add param details, which is fine given the schema fully covers it. A baseline of 4 is appropriate as the description doesn't need to compensate for any gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Get') and resource ('all available test case priorities'), making the purpose specific and understandable. However, it doesn't differentiate from sibling tools like 'get_case_statuses' or 'get_case_types' that also retrieve metadata, leaving room for confusion about when to use this versus similar 'get_' tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. With many sibling tools (e.g., 'get_case_statuses', 'get_case_types'), the description lacks context on prerequisites, typical use cases, or exclusions, leaving the agent to infer usage from the name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_projectB
Get a specific project by ID
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided. Description only says 'get', implying read-only, but fails to disclose return format, permissions, or error behavior. Minimal transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence is efficient and front-loaded. However, it is too brief to add value beyond the name.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema, no annotations, and no behavioral details. A simple fetch tool should mention what it returns or any side effects. Incomplete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema covers 100% of parameters with description 'Project ID'. Description adds no additional meaning beyond what schema already provides. Baseline 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get'), resource ('project'), and method ('by ID'). It effectively distinguishes from sibling tools like 'get_projects'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this vs alternatives like 'get_suite' or 'get_case'. No prerequisites or exclusions mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_projectsB
Get all TestRail projects
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears full burden. It does not disclose whether the operation is read-only, requires authentication, or handles potential large result sets (e.g., pagination). The description is minimal.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, short sentence that is front-loaded and concise. Every word adds value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with no parameters and no output schema, the description adequately communicates the purpose. However, it could mention that the tool returns a list of projects or the format of the output, slightly improving completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
There are no parameters, and schema coverage is 100% (empty schema). Baseline is 3 per instructions. The description adds no additional parameter information beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all TestRail projects' clearly states the action ('Get') and the resource ('all TestRail projects'), distinguishing it from siblings like 'get_project' which retrieves a single project.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For example, if a user needs a single project, they should use 'get_project' instead. Missing context on when 'all' is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_result_fieldsB
Get all available result custom fields
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the basic action without disclosing behavioral traits. It doesn't mention whether this is a read-only operation, if it requires authentication, potential rate limits, or the format of the returned data (e.g., list of field objects). This leaves significant gaps for an agent to understand how to handle the tool effectively.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and wastes no space, making it easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity (a read operation with no parameters) and lack of annotations and output schema, the description is incomplete. It doesn't explain what 'result custom fields' entail (e.g., types, usage) or the return format, leaving the agent without enough context to use the tool effectively beyond the basic action.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here, earning a baseline score of 4 as it doesn't need to compensate for any schema gaps.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('all available result custom fields'), providing a specific purpose. However, it doesn't explicitly differentiate from sibling tools like 'get_case_fields' or 'get_statuses', which might retrieve similar metadata for other entities.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description lacks context about prerequisites (e.g., needing a project or result ID) or comparisons to other metadata-fetching tools in the sibling list, such as 'get_case_fields' or 'get_priorities'.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_resultsC
Get results for a test
| Name | Required | Description | Default |
|---|---|---|---|
| test_id | Yes | Test ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the basic action. It fails to disclose behavioral traits like read-only nature, authentication needs, or response format.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely short (5 words), which is efficient but lacks necessary detail. It is front-loaded but does not earn its place due to missing context.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the single parameter, no output schema, no annotations, and many similar sibling tools, the description is minimally complete. It does not clarify the scope of results or relationship to other tools.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% (one parameter with basic description 'Test ID'), so baseline is 3. The tool description adds no additional meaning to the parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Get results for a test', which is a clear verb+noun combination, but it does not differentiate from sibling tools like get_results_for_case and get_results_for_run, leaving ambiguity about scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives. The description provides no contextual cues, such as when to choose get_results over more specific result retrieval tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_results_for_caseB
Get results for a test case in a run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| case_id | Yes | Case ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It states the operation is a retrieval (read), which implies a safe, non-destructive action. However, it does not disclose any behavioral traits such as pagination, ordering, filtering, or whether results are for all runs or specific ones. For a simple read operation, this is adequate but could be improved.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is to the point and front-loaded. It uses no unnecessary words. While brief, it could benefit from additional context, but it is not verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has only two parameters and no output schema, the description is somewhat complete. However, it lacks details on the return value format or what exactly constitutes 'results'. For a retrieval tool, this information would help the agent use the output correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema covers both parameters with descriptions ('Run ID', 'Case ID'), achieving 100% coverage. The description adds no additional meaning beyond what the schema already provides, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get results for a test case in a run' uses a specific verb ('Get') and clearly identifies the resource ('results for a test case in a run'). It distinguishes itself from sibling tools like 'get_results_for_run' (which gets results for an entire run) and 'get_case' (which gets case details).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description does not provide any guidance on when to use this tool versus alternatives. It lacks explicit context for when this tool should be preferred over siblings like 'get_results_for_run' or 'get_case', and does not mention any prerequisites or limitations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_results_for_runC
Get results for a test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so the description should disclose behavior. It only says 'Get results' implying read-only, but omits details like response format, pagination, or what constitutes a result.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is concise and front-loaded. However, it is too brief and does not fully earn its place due to lack of value-added detail.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description is minimally adequate. It states the purpose but lacks complete guidance on return values or behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the parameter described as 'Run ID'. The description adds no additional meaning beyond what the schema provides, warranting the baseline score.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource, clearly stating it retrieves results for a test run. It distinguishes from siblings like 'get_results_for_case' but could be more explicit about the scope compared to 'get_results'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives like 'get_results' or 'get_results_for_case'. The description does not mention context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_runB
Get a specific test run by ID
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It only repeats the function implied by the name, without disclosing behavioral details such as whether the run is returned, its structure, or error states.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence with no wasted words, but it could be slightly more informative while remaining concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID tool, the description is adequate but lacks information about return values, given no output schema. It does not cover potential errors or limitations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% (one parameter with description). The description adds no further meaning beyond the schema, so baseline score of 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and the resource 'a specific test run by ID', distinguishing it from sibling tools like get_runs (plural) and other get_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives such as get_runs, get_tests, etc. The description does not mention context or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_runsB
Get test runs for a project
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description adds no behavioral details beyond the basic operation. It does not disclose if there are pagination, ordering, or filtering capabilities, nor any side effects or performance implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence that is front-loaded and contains no extraneous information. It is as concise as possible while conveying the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and no output schema, the description is minimally adequate. However, it lacks context about what test runs are or how they relate to other entities, which could be helpful for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of the parameter (project_id) with a description. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get test runs for a project' clearly states the verb (Get) and resource (test runs) with a scope (for a project). However, it does not differentiate from sibling tools like 'get_run' or 'get_tests', which could confuse an agent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. For instance, it does not mention that 'get_run' is for a single run or that this lists all runs, missing an opportunity to clarify selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_sectionA
Get a specific section by ID
| Name | Required | Description | Default |
|---|---|---|---|
| section_id | Yes | Section ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral traits such as required permissions, potential errors, or data scope. It only states the basic action.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence is concise and front-loads the purpose. However, it could include slightly more context without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple get-by-ID tool with one parameter and no output schema, the description is complete enough. It provides the essential information needed for correct tool selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% coverage with a description for 'section_id'. The tool description adds no additional meaning beyond what the schema already provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get a specific section by ID' uses a specific verb and resource, clearly identifying the tool's action. It distinguishes well from sibling tools like 'get_sections' (list) and 'add_section' (create).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'get_sections'. Usage is implied from the description but not clarified.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_sectionsB
Get all sections for a project/suite
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID | |
| suite_id | No | Suite ID (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description carries full burden. Does not disclose behavioral traits such as idempotency, side effects, authentication requirements, or what happens if no sections are found. Only implies a read operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence achieving maximum conciseness. Front-loaded with core purpose, no wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list retrieval tool, description is adequate. However, no output schema means an agent might benefit from knowing return format. Given the many siblings, a note about relationship to get_section or get_suite would improve completeness. Slightly above baseline due to clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions are minimal (Project ID, Suite ID optional). Description adds context that suite_id filters sections within a project, but schema coverage is 100%, so baseline is 3. No additional meaningful detail beyond schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all sections for a project/suite' clearly states the verb (Get) and resource (sections) with scope. It distinguishes from sibling 'get_section' which retrieves a single section, and other get_* tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool vs alternatives. Does not mention that 'get_section' should be used for a specific section, or provide context for choosing over other list tools.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_statusesB
Get all available test result statuses
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden for behavioral disclosure. It only states what the tool does ('Get all available test result statuses') without mentioning permissions, rate limits, response format, or any side effects. This leaves significant gaps for a tool with zero annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It's appropriately sized and front-loaded, making it easy to understand at a glance.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'statuses' entail, the return format, or any behavioral context needed for proper use. For a tool in a complex system with many siblings, more detail would be beneficial.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has 0 parameters with 100% schema description coverage, meaning the schema fully documents the lack of inputs. The description doesn't need to add parameter information, so it meets the baseline of 4 for tools with no parameters, as it doesn't contradict or omit anything.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('all available test result statuses'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'get_case_statuses' or other status-related tools, which would require explicit distinction to earn a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_case_statuses' and 'get_results' that might overlap in context, there's no explicit or implied direction on appropriate usage scenarios or exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_suiteA
Get a specific test suite by ID
| Name | Required | Description | Default |
|---|---|---|---|
| suite_id | Yes | Suite ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must convey behavior fully. It only says 'Get', implying read-only, but does not disclose what happens for invalid IDs (error response), whether the output is the full suite object, or any side effects. This is insufficient for complete transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, concise sentence that immediately states the core purpose. There is no extraneous information; every word is necessary.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (1 required param, no output schema, no annotations, simple resource), the description is mostly complete. It covers what the tool does and the key parameter. The lack of usage guidance is a minor gap, but the description suffices for a straightforward retrieval tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter 'suite_id' described as 'Suite ID'. The description adds 'by ID' but no additional semantics, format, or constraints beyond the schema. Baseline 3 is appropriate since the schema handles the param documentation adequately.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description explicitly states 'Get a specific test suite by ID,' clearly identifying the action (get), resource (test suite), and scope (by ID). It naturally distinguishes from sibling tools like 'get_suites' (lists all) and 'add_suite' (creates).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for retrieving a single test suite by ID, but provides no explicit guidance on when to use this tool versus alternatives, nor any prerequisites or conditions (e.g., 'use when you have the suite_id'). Minimal but adequate for a simple tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_suitesB
Get all test suites for a project
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, and description fails to disclose behavioral traits such as pagination, ordering, or error handling, which are important for a list operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single sentence, no extraneous words. However, could be slightly expanded to cover behavioral context without losing conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list tool with one parameter and no output schema, description minimally covers purpose, but lacks details on return format, potential limits, or error cases.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter 'project_id' described succinctly. Description adds no extra meaning beyond the schema, placing it at baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
Description clearly states 'Get all test suites for a project', using a specific verb and resource, and distinguishes from sibling tools like 'get_suite' (single suite) and 'add_suite'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Description implies usage for listing all suites, but lacks explicit guidance on when to use this vs alternatives like 'get_suite' or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_templatesC
Get all templates for a project
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | Yes | Project ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Get all templates' but does not clarify if this is a read-only operation, what permissions are required, whether results are paginated, or if it returns an error for invalid project IDs. For a retrieval tool with zero annotation coverage, this is a significant gap in transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with no wasted words. It is front-loaded with the core action and resource, making it easy to parse. Every word contributes directly to understanding the tool's purpose, achieving optimal conciseness.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter, no output schema, no annotations), the description is incomplete. It lacks details on behavioral aspects like error handling or result format, and does not guide usage relative to siblings. While concise, it fails to provide sufficient context for an agent to use the tool effectively without additional inference.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage, with 'project_id' documented as 'Project ID'. The description adds minimal value by implying the parameter is used to scope templates to a project, but does not provide additional context like format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate as the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get all templates for a project' clearly states the action (get) and resource (templates), with context (for a project). It distinguishes from siblings like 'get_sections' or 'get_suites' by specifying templates, but does not explicitly differentiate from tools like 'get_case_fields' or 'get_configs' that might also retrieve project-related data. The purpose is specific but lacks explicit sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, such as needing a valid project ID, or exclusions, like whether it retrieves only active templates. With many sibling tools for retrieving various project entities, this lack of context leaves the agent to infer usage based on the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_testB
Get a specific test by ID
| Name | Required | Description | Default |
|---|---|---|---|
| test_id | Yes | Test ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, and the description does not disclose behavioral details such as whether the operation is read-only, what happens if the test_id does not exist, or any side effects. The description simply states the basic retrieval function.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no unnecessary words. It is perfectly concise and front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one parameter, no output schema), the description is minimally adequate but lacks details about the return format, error cases, or behavior when multiple tests might match.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema covers 100% of parameters, so the baseline is 3. The description adds no additional meaning beyond what the schema provides (the param is described as 'Test ID' in the schema, and the description just says 'by ID').
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (get) and the resource (a specific test) and the method (by ID). It distinguishes from sibling tools like get_tests, get_case, etc., by being specific to a single test resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool versus alternatives like get_tests (which retrieves multiple tests) or other get_* tools. There is no mention of prerequisites or context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_testsC
Get tests for a test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, and the description does not disclose behavioral traits such as read-only nature, pagination, sorting, or any side effects. It only states the basic function without clarifying what 'tests' entails or how the data is returned. The description carries the full burden for transparency but falls short.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is very concise, consisting of a single sentence with no redundant words. It is front-loaded with the key information. However, it could benefit from slightly more structure, such as listing parameters or return value, while still being concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (one required parameter, no output schema), the description is minimally complete. It explains what the tool does and what input is needed, but it omits details about the output format, possible empty results, or error conditions. For a tool of this complexity, it is adequate but not thorough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema coverage is 100% with one parameter 'run_id' described as 'Run ID'. The description adds context by stating 'for a test run', which reinforces the parameter's purpose. However, it does not provide additional semantic detail beyond what the schema already provides, so it meets the baseline for high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Get tests for a test run' clearly states the verb ('get') and resource ('tests'), and relates it to a test run. It is specific enough to distinguish from sibling tools like 'get_test' (singular) by implying a collection, but it does not explicitly differentiate from similar tools like 'get_results_for_run'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No usage guidelines are provided. The description does not indicate when to use this tool versus alternatives like 'get_test' for a single test or 'get_results_for_run' for results. There are no explicit when-to-use or when-not-to-use instructions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_userC
Get a user by ID
| Name | Required | Description | Default |
|---|---|---|---|
| user_id | Yes | User ID |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but offers minimal behavioral insight. It doesn't disclose whether this is a read-only operation, what happens if the user ID doesn't exist, authentication needs, rate limits, or response format. 'Get' implies retrieval but lacks operational details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise with a single, clear sentence that front-loads the essential information. There's zero wasted language, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what data is returned, error conditions, or behavioral traits. While simple, it leaves the agent guessing about the operation's full context and results.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with the parameter 'user_id' documented as 'User ID'. The description adds no additional meaning beyond what the schema provides, such as format examples or constraints. Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and resource ('a user by ID'), making the purpose immediately understandable. It distinguishes from siblings like 'get_users' (plural) by specifying singular retrieval, but doesn't explicitly differentiate from 'get_user_by_email' which serves a similar purpose with a different identifier.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. It doesn't mention when to choose 'get_user' over 'get_user_by_email' or 'get_users', nor does it specify prerequisites like authentication requirements or access permissions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_user_by_emailB
Get a user by email address
| Name | Required | Description | Default |
|---|---|---|---|
| Yes | User email address |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It states the tool's function but doesn't disclose behavioral traits like whether it's read-only (implied by 'Get'), what happens if the email isn't found (returns null/error), authentication needs, rate limits, or response format. For a lookup tool with zero annotation coverage, this is a significant gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a simple lookup tool. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's low complexity (single parameter, no output schema, no annotations), the description is minimally adequate but lacks completeness. It doesn't cover behavioral aspects like error handling or response format, which are important for a lookup tool. With no annotations or output schema, more context would be helpful.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the parameter 'email' fully documented in the schema. The description adds no additional parameter semantics beyond what's in the schema (e.g., email format, validation rules). Baseline 3 is appropriate when the schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get') and target resource ('a user'), specifying the lookup method ('by email address'). It distinguishes from sibling 'get_user' (which likely uses ID) and 'get_users' (which lists multiple users), but doesn't explicitly mention these alternatives. The purpose is specific and unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context (when you need to find a user by email rather than ID), but doesn't explicitly state when to use this vs 'get_user' or 'get_users'. No guidance on prerequisites, error conditions, or alternatives is provided. Usage is clear from the name but not elaborated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_usersC
Get all users (optionally filtered by project)
| Name | Required | Description | Default |
|---|---|---|---|
| project_id | No | Project ID (optional, required for non-admins) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It mentions optional filtering but doesn't disclose behavioral traits like pagination, rate limits, authentication requirements, or what 'all users' means in scope (e.g., active only, all roles). For a read operation with no annotations, this leaves significant gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core purpose ('Get all users') and adds optional detail. There is no wasted text, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no output schema, the description is incomplete for a tool that likely returns a list of users. It doesn't explain return format, pagination, error handling, or permissions (e.g., admin vs. non-admin access implied by parameter note but not clarified). For a read tool with potential complexity, this is inadequate.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with the parameter 'project_id' documented in the schema. The description adds minimal value by implying filtering functionality but doesn't provide additional context like format examples or edge cases. Baseline 3 is appropriate given high schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Get' and resource 'users', with optional filtering by project. It distinguishes from sibling 'get_user' (singular) but not explicitly from 'get_user_by_email'. The purpose is specific but could better differentiate from similar tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit guidance on when to use this tool versus alternatives like 'get_user' or 'get_user_by_email'. The description mentions optional filtering but doesn't explain when filtering is appropriate or required. No prerequisites or exclusions are stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
move_cases_to_sectionC
Move test cases to another section
| Name | Required | Description | Default |
|---|---|---|---|
| section_id | Yes | Target section ID | |
| suite_id | Yes | Target suite ID | |
| case_ids | Yes | Array of case IDs to move |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose if this is destructive (e.g., removes cases from original location), requires specific permissions, has side effects, or handles errors, leaving critical gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words, front-loading the core action. It's appropriately sized for the tool's complexity, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks behavioral context, error handling, or result details, failing to compensate for the absence of structured data, which could lead to misuse by an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema fully documents parameters like 'section_id' and 'case_ids'. The description adds no additional meaning beyond implying movement involves these parameters, meeting the baseline for high schema coverage without extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('move') and resources ('test cases to another section'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'copy_cases_to_section' or 'move_section', but it's not vague or tautological.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'copy_cases_to_section' or 'move_section'. It lacks context about prerequisites, such as whether cases must exist or sections be accessible, leaving the agent without usage direction.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
move_sectionC
Move a section to another parent or position
| Name | Required | Description | Default |
|---|---|---|---|
| section_id | Yes | Section ID | |
| parent_id | No | Parent section ID (can be null for root) | |
| after_id | No | Section ID after which to place this section (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('move') but doesn't clarify if this requires specific permissions, whether it's destructive to data, what happens to child elements, or error conditions. This leaves significant gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and wastes no space, making it easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits, error handling, return values, and usage context, leaving the agent with insufficient information to invoke it safely and effectively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents all three parameters. The description adds minimal value by implying the parameters relate to parent/position changes but doesn't provide additional syntax, constraints, or examples beyond the schema's basic descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('move') and resource ('section'), specifying the action of relocating a section to another parent or position. However, it doesn't explicitly differentiate from sibling tools like 'move_cases_to_section', which handles different resources, leaving some ambiguity about scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as 'update_section' for other modifications or 'move_cases_to_section' for different operations. It lacks context about prerequisites, dependencies, or typical scenarios for moving sections.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_caseC
Update an existing test case
| Name | Required | Description | Default |
|---|---|---|---|
| case_id | Yes | Case ID | |
| title | No | Test case title (optional) | |
| template_id | No | Template ID (optional) | |
| type_id | No | Test case type ID (optional) | |
| priority_id | No | Priority ID (optional) | |
| estimate | No | Time estimate (optional) | |
| refs | No | References (optional) | |
| custom_preconds | No | Preconditions (optional) | |
| custom_steps | No | Test steps (optional) | |
| custom_expected | No | Expected result (optional) | |
| custom_steps_separated | No | Separated test steps (optional). Array of step objects with content/expected or shared_step_id |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. 'Update an existing test case' implies a mutation operation but reveals nothing about permissions required, whether changes are reversible, error conditions, or what happens to unspecified fields (partial vs full updates). For a mutation tool with 11 parameters, this leaves critical behavioral aspects undocumented.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that states the core purpose without unnecessary words. It's perfectly front-loaded and wastes no space on redundant information. Every word earns its place in this minimal but complete statement of function.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 11 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't address critical context like authentication requirements, error handling, what constitutes a successful update, or the response format. The agent lacks necessary information to use this tool effectively beyond basic parameter passing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, so all parameters are documented in the schema itself. The description adds no additional parameter semantics beyond what's already in the schema - it doesn't explain relationships between parameters, formatting requirements, or business rules. This meets the baseline for high schema coverage but adds no extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Update') and resource ('an existing test case'), making the purpose immediately understandable. It distinguishes from sibling tools like 'add_case' by specifying 'existing', but doesn't explicitly differentiate from other update tools like 'update_milestone' or 'update_plan' beyond the resource type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (like needing a valid case_id), when to choose this over 'add_case', or how it relates to other update operations in the system. The agent must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_milestoneC
Update an existing milestone
| Name | Required | Description | Default |
|---|---|---|---|
| milestone_id | Yes | Milestone ID | |
| name | No | Milestone name (optional) | |
| description | No | Milestone description (optional) | |
| due_on | No | Due date as UNIX timestamp (optional) | |
| is_completed | No | Mark as completed (optional) | |
| is_started | No | Mark as started (optional) | |
| parent_id | No | Parent milestone ID (optional) | |
| start_on | No | Start date as UNIX timestamp (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. 'Update an existing milestone' implies a mutation operation but reveals nothing about permissions needed, whether changes are reversible, rate limits, error conditions, or what happens to unspecified fields. For a mutation tool with 8 parameters, this is a significant gap in behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with good schema documentation and gets straight to the point without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 8 parameters and no annotations or output schema, the description is inadequate. It doesn't explain what 'update' entails operationally, what the response looks like, error handling, or how it interacts with the system. The agent lacks crucial context for proper tool invocation despite good schema coverage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with all 8 parameters well-documented in the input schema. The description adds no parameter information beyond what's already in the schema, so it meets the baseline of 3 where the schema does the heavy lifting. No additional semantic context is provided.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update an existing milestone' clearly states the action (update) and resource (milestone), but it's vague about what aspects can be updated and doesn't distinguish this tool from sibling update tools like update_case or update_plan. It provides basic purpose but lacks specificity about scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites (e.g., needing an existing milestone), when not to use it, or how it differs from other update tools in the sibling list. The agent must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_planC
Update an existing test plan
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| name | No | Plan name (optional) | |
| description | No | Plan description (optional) | |
| milestone_id | No | Milestone ID (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden for behavioral disclosure. It states this is an update operation (implying mutation), but doesn't cover critical aspects like required permissions, whether changes are reversible, error conditions (e.g., invalid plan_id), or what happens to unspecified fields. This leaves significant gaps for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action ('Update an existing test plan'), making it immediately scannable and appropriately sized for its purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't address behavioral aspects (permissions, side effects), response format, or error handling. While the schema covers parameters well, the overall context for safe and effective use is insufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented in the schema. The description doesn't add any meaning beyond what's in the schema (e.g., it doesn't explain relationships between parameters or provide examples). The baseline score of 3 reflects adequate parameter documentation through the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Update') and resource ('an existing test plan'), making the purpose immediately understandable. It distinguishes from sibling tools like 'add_plan' (create) and 'delete_plan' (remove), but doesn't specify what aspects can be updated beyond the generic term 'update'.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites (e.g., needing an existing plan ID), exclusions, or comparisons to similar tools like 'update_plan_entry' or 'update_case'. The agent must infer usage from the name and schema alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_plan_entryC
Update a test plan entry
| Name | Required | Description | Default |
|---|---|---|---|
| plan_id | Yes | Plan ID | |
| entry_id | Yes | Plan entry ID | |
| name | No | Entry name (optional) | |
| description | No | Entry description (optional) | |
| assignedto_id | No | User ID to assign (optional) | |
| include_all | No | Include all test cases (optional) | |
| case_ids | No | Specific case IDs (optional) | |
| refs | No | References (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. 'Update a test plan entry' implies a mutation operation but provides no information about permissions required, whether changes are reversible, what happens to unspecified fields, error conditions, or response format. For a mutation tool with 8 parameters and no annotation coverage, this leaves significant behavioral gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with good schema documentation and gets straight to the point without unnecessary elaboration.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 8 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't address behavioral aspects like permissions, side effects, or response format, nor does it differentiate from similar sibling tools. The agent would need to make significant assumptions about how to properly use this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 100%, with all 8 parameters clearly documented in the input schema. The description adds no additional parameter information beyond what's already in the schema. According to the scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in the description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update a test plan entry' clearly states the action (update) and resource (test plan entry), which is specific enough to understand the basic function. However, it doesn't differentiate from sibling tools like 'update_plan' or 'update_run_in_plan_entry', leaving ambiguity about what specifically distinguishes this update operation from other update operations in the system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. With multiple sibling update tools (update_case, update_milestone, update_plan, update_run, update_run_in_plan_entry, update_section, update_suite), there's no indication of what makes this tool distinct or when it should be selected over other update operations. The agent must infer usage from the tool name alone.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_runC
Update an existing test run
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| name | No | Run name (optional) | |
| description | No | Run description (optional) | |
| milestone_id | No | Milestone ID (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the full burden. It only states the basic verb (update) without disclosing behavior like what happens if the run_id does not exist, whether the update is partial or full replacement, or any side effects like triggering notifications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single concise sentence that front-loads the purpose. It is appropriate in length but could include slightly more context without becoming verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has four parameters and no output schema, the description is incomplete. It does not mention what the response looks like, error conditions, or that the run must exist. A mutation tool like this requires more behavioral context for safe use.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The description adds no additional meaning about parameters beyond the schema. It does not explain, for example, how milestone_id relates to the run or constraints like uniqueness.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Update an existing test run' clearly states the action and resource, distinguishing it from sibling tools like 'add_run' (create) and 'delete_case' (delete). However, it does not explicitly specify what aspects of a run can be updated, which is partially covered by the input schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, such as requiring a valid run_id or that updates are partial, nor any indication of when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_run_in_plan_entryC
Update a test run inside a plan entry
| Name | Required | Description | Default |
|---|---|---|---|
| run_id | Yes | Run ID | |
| description | No | Run description (optional) | |
| assignedto_id | No | User ID to assign (optional) | |
| include_all | No | Include all test cases (optional) | |
| case_ids | No | Specific case IDs (optional) | |
| refs | No | References (optional) |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. 'Update' implies mutation, but the description doesn't disclose behavioral traits like whether this requires specific permissions, what happens to existing run data not mentioned, whether changes are reversible, or any rate limits. It only states the basic action without operational context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, efficient sentence that front-loads the core action ('Update a test run') and context ('inside a plan entry') with zero wasted words. Every word earns its place in conveying the essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with 6 parameters, no annotations, and no output schema, the description is insufficient. It doesn't address what the tool returns, error conditions, or behavioral implications of updating a run within a plan entry versus standalone. The context signals indicate complexity that isn't matched by descriptive completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional meaning about parameters beyond what's in the schema (e.g., it doesn't explain relationships between 'include_all' and 'case_ids' or what 'refs' format should be). Baseline 3 is appropriate when schema does the heavy lifting.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb ('Update') and resource ('a test run inside a plan entry'), making the purpose immediately understandable. It distinguishes this from generic 'update_run' by specifying the 'inside a plan entry' context, though it doesn't explicitly differentiate from sibling 'update_run' which might update runs outside plan entries.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives like 'update_run' or 'update_plan_entry'. It mentions the context ('inside a plan entry') but doesn't explain why this specific tool is needed or what prerequisites exist for updating runs within plan entries versus standalone runs.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_sectionC
Update an existing section
| Name | Required | Description | Default |
|---|---|---|---|
| section_id | Yes | Section ID | |
| name | No | Section name | |
| description | No | Section description |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fails to disclose behavioral traits such as whether updates are partial or full replacements, permissions needed, or side effects. Only the action 'update' is mentioned, which is insufficient for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is extremely concise at one sentence, but it is not wasteful. However, it could be improved by including key details without sacrificing brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has no output schema and no annotations, the description should explain what happens upon success or failure, or confirm return behavior. It does not, leaving the agent underinformed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 100% description coverage for all three parameters. The description adds no additional meaning beyond what the schema already provides, earning a baseline score of 3.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool updates an existing section, which is a specific verb+resource combination. However, it does not differentiate from sibling update tools like update_case or update_run, as all share similar generic phrasing.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus adding a section, updating other entities, or prerequisites like the section must exist. The description lacks any contextual cues for selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
update_suiteB
Update an existing test suite
| Name | Required | Description | Default |
|---|---|---|---|
| suite_id | Yes | Suite ID | |
| name | No | Suite name | |
| description | No | Suite description |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries full burden. It only says 'Update,' implying mutation, but offers no details on side effects, authorization needs, error handling, or response behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no wasted words. However, it could include more useful information without being significantly longer.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a mutation tool with no output schema and no annotations, the description omits critical context such as return values, error conditions, or whether the operation is idempotent. It is insufficient for an AI agent to fully understand the tool's behavior.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with basic parameter descriptions. The tool description adds no additional meaning beyond the schema, so baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action (update) and the resource (existing test suite), distinguishing it from sibling tools like add_suite (create) and get_suite (read). It is unambiguous and specific.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives, such as update_case or update_run, nor does it mention prerequisites or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
77 tool updates
v1.0.3- First observed
add_attachment_to_case - First observed
add_attachment_to_plan - First observed
add_attachment_to_plan_entry - First observed
add_attachment_to_result - First observed
add_attachment_to_run - First observed
add_case - First observed
add_milestone - First observed
add_plan - First observed
add_plan_entry - First observed
add_result - First observed
add_result_for_case - First observed
add_results - First observed
add_results_for_cases - First observed
add_run - First observed
add_run_to_plan_entry - First observed
add_section - First observed
add_suite - First observed
close_plan - First observed
close_run - First observed
copy_cases_to_section - First observed
delete_attachment - First observed
delete_case - First observed
delete_cases - First observed
delete_milestone - First observed
delete_plan - First observed
delete_plan_entry - First observed
delete_run - First observed
delete_run_from_plan_entry - First observed
delete_section - First observed
get_attachment - First observed
get_attachments_for_case - First observed
get_attachments_for_plan - First observed
get_attachments_for_plan_entry - First observed
get_attachments_for_run - First observed
get_attachments_for_test - First observed
get_case - First observed
get_case_fields - First observed
get_case_statuses - First observed
get_case_types - First observed
get_cases - First observed
get_configs - First observed
get_current_user - First observed
get_history_for_case - First observed
get_milestone - First observed
get_milestones - First observed
get_plan - First observed
get_plans - First observed
get_priorities - First observed
get_project - First observed
get_projects - First observed
get_result_fields - First observed
get_results - First observed
get_results_for_case - First observed
get_results_for_run - First observed
get_run - First observed
get_runs - First observed
get_section - First observed
get_sections - First observed
get_statuses - First observed
get_suite - First observed
get_suites - First observed
get_templates - First observed
get_test - First observed
get_tests - First observed
get_user - First observed
get_user_by_email - First observed
get_users - First observed
move_cases_to_section - First observed
move_section - First observed
update_case - First observed
update_milestone - First observed
update_plan - First observed
update_plan_entry - First observed
update_run - First observed
update_run_in_plan_entry - First observed
update_section - First observed
update_suite
TDQS
Scored across 77 tools
Most tools have distinct purposes targeting specific resources and actions, but there is some overlap that could cause confusion. For example, 'add_result' and 'add_result_for_case' are similar, and multiple attachment tools for different resources might be misselected if the agent isn't careful about the target entity.
Tool names follow a highly consistent verb_noun pattern throughout, with clear actions like add, get, delete, update, copy, move, and close applied to specific nouns. This predictability makes it easy for agents to understand and navigate the toolset.
With 77 tools, the count is excessive for typical MCP server purposes, making it overwhelming and difficult to manage. This large number suggests over-fragmentation of operations that could have been consolidated, such as multiple attachment tools for different resources.
The toolset provides comprehensive CRUD and lifecycle coverage for TestRail's domain, including projects, suites, cases, runs, plans, results, attachments, and users. There are no obvious gaps, with tools for creation, retrieval, updates, deletions, and specialized operations like moving or copying cases.
Maintenance
Related MCP Connectors
Direct access to Cypress tests results and accessibility reports in your AI workflow.
Manage test suites, run tests, view results, and automate QA workflows via AI with testRigor.
AI-callable tools for API mocking, testing, monitoring, security, and automation.
Create and manage MeisterTask projects, tasks, and notes from your AI assistant.
Related MCP Servers
- AlicenseBqualityDmaintenanceEnables AI assistants to interact with TestRail test management systems through comprehensive API integration. Supports retrieving and updating test cases, projects, suites, runs, and results, plus adding attachments and managing test data through natural language commands.1812 npmMIT
- AlicenseNot gradedqualityDmaintenanceEnables AI assistants to interact with Testmo test management platform for creating, reading, updating, and deleting test cases, managing folders, and organizing test runs through natural language.4MIT
- AlicenseAqualityFmaintenanceEnables management of TestRail projects, test cases, runs, and results directly through MCP-supported clients. It provides a comprehensive set of tools to interact with the TestRail API for seamless test cycle management within AI environments.42743 npm44MIT
- AlicenseBqualityDmaintenanceEnables AI assistants to interact with TestRail test management system, supporting full CRUD operations on projects, suites, sections, test cases, runs, results, plans, and milestones.351,504 npm1MIT