Skip to main content
Glama
TenBarrel6

TestRail MCP Server

by TenBarrel6

TestRail MCP Server

A Model Context Protocol (MCP) server for seamless TestRail integration. This server enables AI assistants like Claude, Cursor, and other MCP-compatible clients to interact directly with your TestRail instance.

Features

  • 🔐 Secure Authentication - API key-based authentication with TestRail

  • 📊 Full TestRail API Coverage - Support for TestRail API v9.7.2

  • 🚀 Easy Integration - Works with Cursor, Claude Desktop, and other MCP clients

  • 📦 Zero Configuration - Run directly with npx

Related MCP server: Testmo MCP Server

Installation

Global Installation

npm i -g @tenbarrel6/testrail-mcp

Local Installation

npm i @tenbarrel6/testrail-mcp

Using npx

No installation required - run directly in downloaded github repository:

npx @tenbarrel6/testrail-mcp

Configuration

Environment Variables

Create a .env file in your project root or set environment variables:

TESTRAIL_URL=https://your-domain.testrail.io
TESTRAIL_USERNAME=your-email@example.com
TESTRAIL_API_KEY=your-api-key-here

Cursor IDE Configuration

Add to your Cursor settings (.cursor/mcp.json or global settings):

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@tenbarrel6/testrail-mcp"],
      "env": {
        "TESTRAIL_URL": "https://your-company.testrail.io",
        "TESTRAIL_USERNAME": "your-email@company.com",
        "TESTRAIL_API_KEY": "your-api-key-here"
      }
    }
  }
}

Claude Desktop Configuration

Add to your Claude Desktop config (~/Library/Application Support/Claude/claude_desktop_config.json on macOS):

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@tenbarrel6/testrail-mcp"],
      "env": {
        "TESTRAIL_URL": "https://your-company.testrail.io",
        "TESTRAIL_USERNAME": "your-email@company.com",
        "TESTRAIL_API_KEY": "your-api-key-here"
      }
    }
  }
}

Getting Your TestRail API Key

  1. Log in to your TestRail instance

  2. Go to My Settings (click your name in the top right)

  3. Navigate to the API Keys tab

  4. Click Add Key to generate a new API key

  5. Copy and save the key securely

Available Tools

Projects

Tool

Description

get_projects

Get all TestRail projects

get_project

Get a specific project by ID

Test Suites

Tool

Description

get_suites

Get all test suites for a project

get_suite

Get a specific test suite by ID

add_suite

Create a new test suite

update_suite

Update an existing test suite

Sections

Tool

Description

get_sections

Get all sections for a project/suite

get_section

Get a specific section by ID

add_section

Create a new section

update_section

Update an existing section

delete_section

Delete a section

move_section

Move a section to another parent or position

Test Cases

Tool

Description

get_cases

Get test cases for a project/suite

get_case

Get a specific test case by ID

add_case

Create a new test case

update_case

Update an existing test case

delete_case

Delete a test case

delete_cases

Delete multiple test cases

copy_cases_to_section

Copy test cases to another section

move_cases_to_section

Move test cases to another section

get_case_types

Get all available test case types

get_case_fields

Get all available test case fields

get_history_for_case

Get the edit history for a test case

Test Runs

Tool

Description

get_runs

Get test runs for a project

get_run

Get a specific test run by ID

add_run

Create a new test run

update_run

Update an existing test run

close_run

Close a test run

delete_run

Delete a test run

Tests

Tool

Description

get_tests

Get tests for a test run

get_test

Get a specific test by ID

Results

Tool

Description

get_results

Get results for a test

get_results_for_case

Get results for a test case in a run

get_results_for_run

Get all results for a test run

add_result

Add a test result

add_result_for_case

Add a test result for a specific case in a run

add_results

Add multiple test results by test IDs

add_results_for_cases

Add multiple test results for cases in a run

Test Plans

Tool

Description

get_plans

Get test plans for a project

get_plan

Get a specific test plan by ID

add_plan

Create a new test plan

update_plan

Update an existing test plan

close_plan

Close a test plan

delete_plan

Delete a test plan

add_plan_entry

Add test runs to a test plan

update_plan_entry

Update a test plan entry

delete_plan_entry

Delete a test plan entry

add_run_to_plan_entry

Add a test run to an existing plan entry

update_run_in_plan_entry

Update a test run inside a plan entry

delete_run_from_plan_entry

Delete a test run from a plan entry

Milestones

Tool

Description

get_milestones

Get milestones for a project

get_milestone

Get a specific milestone by ID

add_milestone

Create a new milestone

update_milestone

Update an existing milestone

delete_milestone

Delete a milestone

Users

Tool

Description

get_users

Get all users (optionally filtered by project)

get_user

Get a user by ID

get_user_by_email

Get a user by email address

get_current_user

Get the current authenticated user

Statuses & Priorities

Tool

Description

get_statuses

Get all available test result statuses

get_case_statuses

Get all available test case statuses (Enterprise)

get_priorities

Get all available test case priorities

Templates & Configurations

Tool

Description

get_templates

Get all templates for a project

get_configs

Get all configurations for a project

get_result_fields

Get all available result custom fields

Attachments

Tool

Description

add_attachment_to_case

Add an attachment to a test case

add_attachment_to_result

Add an attachment to a test result

add_attachment_to_run

Add an attachment to a test run

add_attachment_to_plan

Add an attachment to a test plan

add_attachment_to_plan_entry

Add an attachment to a test plan entry

get_attachment

Get/download an attachment by ID

get_attachments_for_case

Get all attachments for a test case

get_attachments_for_test

Get all attachments for a test

get_attachments_for_run

Get all attachments for a test run

get_attachments_for_plan

Get all attachments for a test plan

get_attachments_for_plan_entry

Get all attachments for a test plan entry

delete_attachment

Delete an attachment

Usage Examples

Once configured, you can interact with TestRail through your AI assistant:

Get all projects

"List all TestRail projects"

Create a test case

"Create a new test case in section 123 with title 'Verify login functionality'"

Add test results

"Add a passed result for test case 456 in run 789 with comment 'All checks passed'"

Get test run results

"Show me all results for test run 101"

Test Result Status IDs

When adding results, use these standard status IDs:

Status ID

Status

1

Passed

2

Blocked

3

Untested

4

Retest

5

Failed

Development

Building from Source

# Clone the repository
git clone https://github.com/TenBarrel6/testrail-mcp.git
cd testrail-mcp

# Install dependencies
npm install

# Build
npm run build

# Run in development mode
npm run dev

Scripts

  • npm run build - Compile TypeScript to JavaScript

  • npm run start - Run the compiled server

  • npm run dev - Run in development mode with hot reload

Requirements

  • Node.js 18+

  • TestRail instance with API access enabled

  • Valid TestRail API key

Troubleshooting

"Not authenticated" Error

Ensure your environment variables are correctly set:

  • TESTRAIL_URL should include the protocol (https://)

  • TESTRAIL_USERNAME should be your email address

  • TESTRAIL_API_KEY should be a valid API key (not your password)

Connection Issues

  • Verify your TestRail URL is accessible

  • Check if API access is enabled in your TestRail administration settings

  • Ensure your user has appropriate permissions

MCP Client Not Connecting

  • Restart your MCP client (Cursor/Claude Desktop)

  • Verify the configuration JSON syntax is valid

  • Check that npx is available in your PATH

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

  1. Fork the repository

  2. Create your feature branch (git checkout -b feature/amazing-feature)

  3. Commit your changes (git commit -m 'Add some amazing feature')

  4. Push to the branch (git push origin feature/amazing-feature)

  5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Author

Ruslan Sapun

Available Tools

77 tools
add_attachment_to_caseC

Add an attachment to a test case

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesCase ID
file_pathYesPath to the file to upload

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. 'Add an attachment' implies a write/mutation operation, but the description doesn't disclose behavioral traits like required permissions, file size limits, supported file types, whether this overwrites existing attachments, or what happens on success/failure. This is inadequate for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for what it communicates.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns, error conditions, or important behavioral constraints. Given the complexity of file attachment operations and the lack of structured metadata, the description should provide more context about how the operation works.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters (case_id and file_path). The description doesn't add any meaning beyond what the schema provides - it doesn't explain what constitutes a valid case_id or file_path format. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Add an attachment to a test case' clearly states the action (add) and target resource (attachment to test case). It's specific but doesn't distinguish from sibling attachment tools like 'add_attachment_to_plan' or 'add_attachment_to_result' which have similar structures but different targets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. With multiple attachment-related siblings (add_attachment_to_plan, add_attachment_to_result, etc.), the description doesn't explain that this is specifically for test cases versus other entities. No prerequisites or exclusions are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_attachment_to_planC

Add an attachment to a test plan

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
file_pathYesPath to the file to upload

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Add an attachment' implies a write/mutation operation, it doesn't specify permissions required, side effects (e.g., file size limits, supported formats), or response behavior (e.g., success/failure indicators). This leaves significant gaps for an agent to understand how to use it safely and effectively.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words, directly stating the tool's purpose. It is appropriately sized and front-loaded, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, and the tool being a mutation operation (adding attachments), the description is incomplete. It doesn't address behavioral aspects like error handling, return values, or system constraints, which are crucial for an agent to invoke it correctly in a test management context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with clear parameter descriptions in the schema ('Plan ID' and 'Path to the file to upload'). The description adds no additional semantic context beyond what the schema provides, such as format examples or constraints, so it meets the baseline for high schema coverage without compensating value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Add an attachment') and target resource ('to a test plan'), which is specific and unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_plan_entry', which have similar structures but target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'add_attachment_to_case' or 'add_attachment_to_plan_entry', nor does it mention prerequisites, constraints, or exclusions. It merely states what the tool does without contextual usage information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_attachment_to_plan_entryC

Add an attachment to a test plan entry

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
entry_idYesPlan entry ID
file_pathYesPath to the file to upload

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action is to 'Add an attachment', implying a write/mutation operation, but doesn't specify permissions required, side effects (e.g., if it overwrites existing attachments), rate limits, or error conditions. This leaves significant gaps for an agent to understand the tool's behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It's appropriately sized for a tool with a straightforward action and well-documented schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete. It doesn't address behavioral aspects like mutation effects, error handling, or return values, which are critical for a write operation. For a tool with 3 parameters and no structured safety hints, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with clear parameter descriptions in the schema (plan_id, entry_id, file_path). The description doesn't add any meaning beyond this, such as explaining relationships between parameters (e.g., entry_id must belong to plan_id) or file format constraints. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Add an attachment') and target resource ('to a test plan entry'), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_plan', which have similar naming patterns but target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'add_attachment_to_case' or 'add_attachment_to_plan'. It lacks context about prerequisites (e.g., needing an existing plan entry) or exclusions (e.g., not for adding attachments to other entities).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_attachment_to_resultC

Add an attachment to a test result

ParametersJSON Schema
NameRequiredDescriptionDefault
result_idYesResult ID
file_pathYesPath to the file to upload

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It states the action ('Add') but doesn't disclose behavioral traits like required permissions, whether this is a write operation, potential side effects (e.g., file size limits, supported formats), or error handling. This is inadequate for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that this is a mutation tool with no annotations and no output schema, the description is incomplete. It lacks crucial context like behavioral details (e.g., permissions, side effects), usage guidelines relative to siblings, and information about return values or errors, leaving significant gaps for an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with clear parameter names and descriptions ('Result ID', 'Path to the file to upload'). The description doesn't add any meaning beyond this, such as format details or constraints, but the schema provides a solid baseline, so a score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Add an attachment') and target resource ('to a test result'), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_run', which have similar structures but different targets.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'add_attachment_to_case' or 'add_attachment_to_run'. It lacks context about prerequisites, such as whether the result must exist or be in a specific state, and doesn't mention any exclusions or complementary tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_attachment_to_runC

Add an attachment to a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
file_pathYesPath to the file to upload

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Add' implies a write/mutation operation, the description doesn't address important behavioral aspects like required permissions, file size limits, supported file types, whether this operation is idempotent, or what happens on failure. For a mutation tool with zero annotation coverage, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that states the tool's purpose without unnecessary words. It's appropriately sized for a simple tool and front-loads the essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is inadequate. It doesn't explain what happens after the attachment is added, what the response looks like, error conditions, or important constraints. Given the complexity of file upload operations and the lack of structured metadata, the description should provide more contextual information.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters (run_id and file_path) having clear descriptions in the schema. The description doesn't add any additional semantic context beyond what the schema already provides about these parameters, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Add an attachment') and target resource ('to a test run'), providing a specific verb+resource combination. However, it doesn't explicitly differentiate from sibling tools like 'add_attachment_to_case' or 'add_attachment_to_plan', which have similar structures but target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With multiple attachment-related tools in the sibling list (add_attachment_to_case, add_attachment_to_plan, add_attachment_to_plan_entry, add_attachment_to_result), there's no indication of when this specific tool is appropriate versus those other attachment tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_caseC

Create a new test case

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesSection ID
titleYesTest case title
template_idNoTemplate ID (optional)
type_idNoTest case type ID (optional)
priority_idNoPriority ID (optional)
estimateNoTime estimate (optional)
milestone_idNoMilestone ID (optional)
refsNoReferences/Requirements (optional)
custom_precondsNoPreconditions (optional)
custom_stepsNoTest steps (optional)
custom_expectedNoExpected result (optional)
custom_autostatNoAutomation Status (optional)
custom_steps_separatedNoSeparated test steps (optional). Array of step objects with content/expected or shared_step_id

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. 'Create a new test case' implies a write operation but reveals nothing about permissions needed, side effects, error conditions, or what happens on success (e.g., whether a case ID is returned). This leaves significant gaps for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple creation operation and front-loads the essential action without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 13 parameters, no annotations, and no output schema, the description is insufficient. It doesn't explain what happens after creation (e.g., returns a case ID), error handling, or system constraints. The high parameter count and lack of behavioral context make this description incomplete for effective tool use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 13 parameters. The description adds no parameter-specific information beyond what's in the schema, which meets the baseline for high coverage but doesn't provide additional context like typical values or relationships between parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a new test case' clearly states the verb ('Create') and resource ('test case'), making the purpose immediately understandable. However, it doesn't differentiate this tool from other creation tools like 'add_section' or 'add_milestone' beyond the resource type, which prevents a perfect score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With many sibling tools available (e.g., 'update_case', 'delete_case', 'get_cases'), there's no indication of prerequisites, appropriate contexts, or distinctions from related operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_milestoneC

Create a new milestone

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
nameYesMilestone name
descriptionNoMilestone description (optional)
due_onNoDue date as UNIX timestamp (optional)
parent_idNoParent milestone ID for sub-milestones (optional)
refsNoReferences (optional)
start_onNoStart date as UNIX timestamp (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose permissions required, whether creation is idempotent, error handling, or what happens on success (e.g., returns a milestone ID). This is inadequate for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with no annotations and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., the new milestone object or ID), error conditions, or dependencies like project existence. Given the complexity of 7 parameters and mutation nature, more context is needed for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents all 7 parameters, including optionality and data types. The description adds no additional meaning beyond implying creation, which is already clear from the tool name. Baseline 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a new milestone' clearly states the action (create) and resource (milestone), which is specific and unambiguous. However, it doesn't differentiate from sibling tools like 'add_case' or 'add_suite' beyond the resource type, missing explicit distinction in scope or hierarchy.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'update_milestone' or other 'add_' tools. The description lacks context about prerequisites, such as needing a valid project, or exclusions, leaving the agent to infer usage from the input schema alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_planC

Create a new test plan

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
nameYesPlan name
descriptionNoPlan description (optional)
milestone_idNoMilestone ID (optional)
entriesNoArray of plan entries/test runs (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but offers minimal behavioral insight. It states 'Create' implying a write operation, but doesn't disclose permissions needed, side effects, error conditions, or what happens on success (e.g., returns a plan ID).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable and appropriately sized for its purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with 5 parameters, no annotations, and no output schema, the description is inadequate. It doesn't explain what a 'test plan' is in this context, what happens after creation, or how it relates to other entities like projects or milestones mentioned in parameters.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are documented in the schema. The description adds no additional parameter context beyond implying creation of a 'test plan', which aligns with the schema but doesn't provide extra semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create') and resource ('new test plan'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'add_plan_entry' or 'update_plan', but it's not misleading or tautological.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'update_plan' or 'add_plan_entry'. It doesn't mention prerequisites, dependencies, or contextual triggers for creation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_plan_entryC

Add test runs to a test plan

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
suite_idYesSuite ID
nameNoEntry name (optional)
descriptionNoEntry description (optional)
assignedto_idNoUser ID to assign (optional)
include_allNoInclude all test cases (optional)
case_idsNoSpecific case IDs (optional)
config_idsNoConfiguration IDs (optional)
refsNoReferences (optional)
runsNoArray of test runs with configurations (optional)

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. 'Add test runs to a test plan' implies a write operation but doesn't specify permissions needed, whether it's idempotent, what happens on failure, or the format of results. For a mutation tool with 10 parameters, this minimal description fails to provide necessary behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence with zero wasted words. It's appropriately sized for a tool name that's self-explanatory ('add_plan_entry'), though this conciseness comes at the cost of completeness. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (10 parameters, mutation operation, no annotations, no output schema), the description is incomplete. It doesn't explain what a 'plan entry' is, how test runs are structured, what happens when adding runs, or what the tool returns. For a tool with this many parameters and no structured guidance, more context is needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional parameter semantics beyond implying 'test runs' relate to the 'runs' parameter. This meets the baseline of 3 since the schema does the heavy lifting, but the description doesn't enhance understanding of how parameters interact.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Add test runs to a test plan' clearly states the action (add) and target (test runs to a test plan), which is adequate. However, it doesn't differentiate from sibling tools like 'add_run_to_plan_entry' or 'add_run', leaving ambiguity about whether this creates new runs or adds existing ones to a plan. The purpose is clear but lacks sibling distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With many sibling tools like 'add_run', 'add_run_to_plan_entry', and 'add_plan', there's no indication of prerequisites, context, or exclusions. This leaves the agent guessing about appropriate use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_resultC

Add a test result

ParametersJSON Schema
NameRequiredDescriptionDefault
test_idYesTest ID
status_idYesStatus ID (1=Passed, 5=Failed, etc.)
commentNoComment (optional)
elapsedNoTime elapsed (optional)
defectsNoDefect IDs (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description should disclose behavioral traits like side effects, required permissions, or return values. It only states the basic function without any additional context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no unnecessary words. It is concise and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of output schema and annotations, the description is too minimal for a tool with 5 parameters. It fails to explain the result of the operation or any constraints.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, with all parameters described. The description adds no extra meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Add' and the resource 'a test result', which is distinct from siblings like 'add_case' or 'add_result_for_case'. However, it could be more specific by mentioning it's for a test case result.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool vs alternatives such as 'add_result_for_case' or 'add_results_for_cases'. The description lacks context for proper selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_result_for_caseB

Add a test result for a specific case in a run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
case_idYesCase ID
status_idYesStatus ID
commentNoComment (optional)
elapsedNoTime elapsed (optional)

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description only states the action. It does not disclose behavioral traits like whether it creates or updates an existing result, required permissions, or side effects. For a write operation, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that front-loads the key information. Every word is necessary and contributes to understanding the tool's purpose. No redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description lacks details about the tool's output (no output schema), edge cases (e.g., duplicate results), prerequisites, or how parameters like 'elapsed' are formatted. Given 5 parameters and no annotations, it is minimally complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the baseline is 3. The description adds no additional meaning beyond what the schema already provides for each parameter. It does not clarify formats or valid values (e.g., for 'elapsed' or 'status_id').

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Add'), the resource ('test result'), and the context ('for a specific case in a run'). It effectively distinguishes from sibling tools like 'add_result' and 'add_results_for_cases'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is for adding a result to a specific case in a run, but does not provide explicit guidance on when to use it over alternatives (e.g., 'add_result' or 'add_results_for_cases'). No when-not-to-use or prerequisites are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_resultsC

Add multiple test results by test IDs

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
resultsYesArray of results

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Add' implies a write operation, but the description doesn't specify whether this requires authentication, what happens on success or error (e.g., partial updates), or if there are rate limits. It also doesn't clarify if 'add' means create new entries or update existing ones, leaving behavioral traits ambiguous. This is a significant gap for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and resource, making it easy to parse. However, it could be slightly more informative by clarifying the scope of 'add' (e.g., 'create' or 'append'), but as-is, it avoids redundancy and waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a bulk write operation with no annotations and no output schema, the description is inadequate. It doesn't address key aspects like authentication needs, error handling, or what the tool returns (e.g., success status, IDs of added results). For a tool that modifies data and has many similar siblings, more context is needed to ensure correct usage and avoid confusion.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with clear documentation for 'run_id' and 'results' (including nested properties like test_id, status_id). The description adds minimal value beyond the schema, mentioning 'test IDs' which aligns with the 'test_id' parameter but doesn't explain the relationship between test IDs and results or provide additional context like format examples. With high schema coverage, the baseline score of 3 is appropriate as the schema does most of the work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Add multiple test results by test IDs' clearly states the action (add) and resource (test results), but it's somewhat vague about what 'add' means in this context. It doesn't specify whether this creates new results, appends to existing ones, or updates them. Compared to sibling tools like 'add_result' (singular) and 'add_results_for_cases', it distinguishes itself by focusing on test IDs rather than cases, but the distinction isn't explicitly articulated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'add_result' (singular), 'add_results_for_cases', or 'add_result_for_case'. It doesn't mention prerequisites, such as needing an existing run or test, or specify scenarios where bulk addition by test IDs is preferred over other methods. Without this context, an agent might struggle to choose the correct tool among similar siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_results_for_casesB

Add multiple test results for cases in a run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
resultsYesArray of results

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, and the description only states the action ('add') without disclosing side effects, permissions needed, or behavior on errors (e.g., if run_id is invalid). The burden is on the description, which is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence with no unnecessary words. It conveys the essential purpose efficiently, though it could benefit from a bit more detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has nested parameters without full documentation (e.g., what status_id values are valid) and no output schema. The description does not explain the structure of the results array or provide sufficient context for correct usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with basic descriptions ('Run ID', 'Array of results'). The description does not add further meaning beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Add' and the resource 'multiple test results for cases in a run', distinguishing it from sibling tools like add_result (single) and add_result_for_case (single for case).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives. Sibling tools exist for adding single results, but the description does not explicitly differentiate or mention when to prefer this bulk operation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_runB

Create a new test run

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
suite_idNoSuite ID (optional)
nameYesRun name
descriptionNoRun description (optional)
milestone_idNoMilestone ID (optional)
assignedto_idNoUser ID to assign (optional)
include_allNoInclude all test cases (optional)
case_idsNoSpecific case IDs (optional)

TDQS

B3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description only says 'Create a new test run', which implies mutation, but lacks detail on side effects, authorization needed, or what happens upon success. Without annotations, the description should compensate but does not.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, but it is too brief. It wastes no words but also provides minimal information. A single sentence suffices for the purpose, but it could be more informative without losing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 8 parameters, no output schema, and no annotations, the description is insufficient. It does not explain return values, error states, or how parameters like include_all and case_ids interact. The agent may lack context for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for its 8 parameters, so the description adds no additional meaning. The baseline of 3 is appropriate as the schema already documents parameter semantics adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Create' and resource 'new test run', which is specific and unambiguous. It succinctly indicates the tool's function without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool vs alternatives like update_run or add_case. There is no mention of prerequisites or context that would help the agent decide appropriately.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_run_to_plan_entryC

Add a test run to an existing plan entry

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
entry_idYesPlan entry ID
config_idsYesConfiguration IDs
descriptionNoRun description (optional)
assignedto_idNoUser ID to assign (optional)
include_allNoInclude all test cases (optional)
case_idsNoSpecific case IDs (optional)
refsNoReferences (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. While 'add' implies a write/mutation operation, the description doesn't specify permissions required, whether this creates a new run or associates existing runs, what happens on success/failure, rate limits, or side effects. For a tool that modifies data with 8 parameters, this is insufficient behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence that efficiently communicates the core function. There's no wasted verbiage, repetition, or unnecessary elaboration. It's appropriately sized for the tool's complexity and gets straight to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 8 parameters, no annotations, and no output schema, the description is incomplete. It doesn't address what the tool returns, error conditions, authentication requirements, or how it differs from similar tools. The description provides only the most basic functional statement without the context needed for effective use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all 8 parameters with basic descriptions. The tool description adds no additional parameter information beyond what's in the schema - it doesn't explain relationships between parameters (like how 'include_all' interacts with 'case_ids'), parameter constraints, or practical usage examples. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('add a test run') and target resource ('to an existing plan entry'), providing specific verb+resource pairing. However, it doesn't differentiate from sibling tools like 'add_run' or 'add_run_to_plan_entry' vs 'update_run_in_plan_entry', which would require more specificity about what distinguishes this particular addition operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With many sibling tools like 'add_run', 'add_plan_entry', 'update_run_in_plan_entry', and 'delete_run_from_plan_entry', there's no indication of prerequisites, appropriate contexts, or when other tools might be more suitable. The description simply states what it does without usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_sectionC

Create a new section

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
suite_idNoSuite ID (optional)
parent_idNoParent section ID (optional)
nameYesSection name
descriptionNoSection description (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only says 'Create a new section' without mentioning effects, permissions, idempotency, or error conditions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, unambiguous sentence. It is as concise as possible with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description should mention that the created section object is returned. It also lacks context on how this tool fits among siblings (e.g., when to use add_section vs add_suite). The description is inadequate for an agent to invoke correctly without prior knowledge.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, so each parameter is documented. The tool description adds no extra meaning beyond the schema, such as explaining the relationship between suite_id and parent_id.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a new section' clearly states the verb (create) and resource (section). It distinguishes from siblings like add_case or add_suite by specifying the resource type, but does not elaborate on scope or nuances.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as add_suite or update_section. No context about prerequisites or conditions is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_suiteB

Create a new test suite

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
nameYesSuite name
descriptionNoSuite description (optional)

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, yet the description does not disclose any behavioral traits beyond the basic action, such as required project existence or uniqueness constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single efficient sentence, but it could include more context without becoming verbose, slightly limiting its usefulness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Without an output schema or annotations, the description fails to explain return values or usage context, leaving a significant gap in completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the tool description adds no additional meaning beyond what the schema already provides, warranting a baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a new test suite' clearly states the specific verb and resource, distinguishing it from sibling tools like add_case or add_run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for creating suites but provides no explicit guidance on when to use it versus alternatives, nor any exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

close_planC

Close a test plan

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. 'Close a test plan' implies a mutation operation, but it doesn't specify whether this is reversible, what permissions are required, if it affects associated runs/cases, or what the outcome looks like. For a destructive-sounding tool with zero annotation coverage, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately understandable. For such a simple tool, this brevity is appropriate and effective.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity (a mutation operation with potential side effects), lack of annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like idempotency, error conditions, or return values. For a 'close' operation in a test management context, more context is needed to use it safely and effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage (the 'plan_id' parameter is documented as 'Plan ID'), so the baseline is 3. The description adds no additional parameter context beyond what the schema provides—it doesn't explain where to find the plan_id, format constraints, or validation rules. This meets minimum viability but offers no extra semantic value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Close a test plan' clearly states the verb ('close') and resource ('test plan'), providing basic purpose. However, it doesn't differentiate from sibling tools like 'close_run' or explain what 'closing' means in this context (e.g., marking as completed vs. archiving). The purpose is understandable but lacks specificity compared to alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether the plan must be open), exclusions (e.g., cannot close if active runs exist), or related tools like 'update_plan' for status changes. With many sibling tools for managing plans, this omission leaves the agent guessing about appropriate usage contexts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

close_runB

Close a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. The description does not disclose what closing a run entails (e.g., archiving, preventing further results, required permissions).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no extraneous information, perfectly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description fails to explain return values, side effects, or error states (e.g., closing an already closed run). Incomplete for a mutation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter description ('Run ID') is clear. The tool description adds no extra meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('close') and the resource ('test run'), distinguishing it from sibling tools like add_run, update_run, get_run.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives, such as update_run or deleting a run. Lacks context about prerequisites or post-conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

copy_cases_to_sectionC

Copy test cases to another section

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesTarget section ID
case_idsYesArray of case IDs to copy

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'copy' but does not clarify if this creates duplicates, retains originals, requires permissions, or handles errors. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its effects and requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste—'Copy test cases to another section'—front-loading the core action. It is appropriately sized for the tool's complexity, with no redundant or verbose elements.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool involves mutation (copying cases) with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits (e.g., duplication effects, permissions), error handling, or return values. For a tool with two required parameters and potential side effects, more context is needed to ensure safe and correct usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with clear descriptions for 'section_id' (target section ID) and 'case_ids' (array of case IDs to copy). The description adds no additional semantic context beyond what the schema provides, such as format details or constraints. With high schema coverage, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'copy' and the resource 'test cases', specifying the destination 'to another section'. It distinguishes from the sibling 'move_cases_to_section' by implying copying rather than moving, though this distinction could be more explicit. The purpose is specific and actionable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'move_cases_to_section' or 'add_case'. It lacks context about prerequisites, such as whether the source cases or target section must exist, or any constraints on copying. Usage is implied only by the action described.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_attachmentC

Delete an attachment

ParametersJSON Schema
NameRequiredDescriptionDefault
attachment_idYesAttachment ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Delete' implies a destructive mutation, but the description doesn't specify whether this action is reversible, requires specific permissions, has side effects (e.g., on linked cases or runs), or provides confirmation. This is a significant gap for a mutation tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with zero wasted words, making it highly concise and front-loaded. It efficiently communicates the core action without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive mutation tool with no annotations and no output schema, the description is incomplete. It lacks critical context like behavioral traits (e.g., irreversibility, permissions), usage guidelines, or output expectations. Given the complexity and risk of deletion, this minimal description is inadequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with the single parameter 'attachment_id' documented as 'Attachment ID'. The description doesn't add any meaning beyond this, such as format examples or sourcing guidance. With high schema coverage, the baseline score of 3 is appropriate as the schema handles the parameter documentation adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete an attachment' clearly states the action (delete) and the resource (attachment), making the tool's purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'delete_case' or 'delete_plan', which follow the same pattern but target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an attachment ID), exclusions, or relationships with sibling tools like 'get_attachment' for verification or attachment-specific tools for context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_caseB

Delete a test case

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesCase ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, and the description only states 'delete' without detailing whether deletion is permanent, cascading, or requires authentication.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence is concise and to the point, but could include a bit more context without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter, the description is minimally complete but lacks details on irreversibility or effects on related data.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with parameter description 'Case ID', which already conveys the meaning. The description adds no extra insight beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a test case' clearly states the action and resource, distinguishing it from sibling tools like add_case and update_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives (e.g., update_case to deactivate instead). Lacks when-not-to-use or prerequisite conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_casesC

Delete multiple test cases

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
suite_idNoSuite ID (required for multi-suite projects)
case_idsYesArray of case IDs to delete
softNoSet to 1 to preview deletion without executing (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states the action ('delete') without disclosing critical behavioral traits. It doesn't mention whether deletion is permanent or reversible, what permissions are required, whether it affects related data (like attachments or results), or what happens on success/failure. For a destructive operation, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient phrase with zero wasted words. It's front-loaded with the core action and resource, making it immediately understandable despite its brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It lacks behavioral context (permissions, reversibility, side effects), usage differentiation from siblings, and any information about return values or error conditions. The schema covers parameters well, but the description doesn't compensate for other gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional semantic context about parameters beyond implying 'multiple' cases via 'case_ids'. It doesn't explain relationships between parameters (e.g., when 'suite_id' is needed) or provide examples. Baseline 3 is appropriate given the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete multiple test cases' clearly states the verb (delete) and resource (test cases), and specifies 'multiple' to indicate batch operation. However, it doesn't differentiate from sibling 'delete_case' which presumably deletes single cases, leaving some ambiguity about when to choose one over the other.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives like 'delete_case' (for single deletions) or other deletion tools in the sibling list. The description mentions 'multiple' but doesn't explicitly contrast with single-case deletion or provide context about prerequisites or constraints.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_milestoneC

Delete a milestone

ParametersJSON Schema
NameRequiredDescriptionDefault
milestone_idYesMilestone ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It states 'Delete a milestone,' implying a destructive mutation, but doesn't disclose behavioral traits such as permissions required, whether deletion is permanent or reversible, error handling, or side effects. This is a significant gap for a destructive tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise ('Delete a milestone') and front-loaded with the core action. It wastes no words, making it efficient for quick understanding, though this brevity contributes to gaps in other dimensions.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's destructive nature, no annotations, and no output schema, the description is incomplete. It fails to address critical context like what happens post-deletion, success/error responses, or dependencies, making it inadequate for safe and effective use by an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter (milestone_id), so the schema already documents it adequately. The description doesn't add parameter details, but with 0 parameters needing extra explanation, a baseline of 4 is appropriate as it doesn't detract from clarity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a milestone' clearly states the action (delete) and resource (milestone), which is adequate. However, it doesn't differentiate this tool from other deletion tools in the sibling list (e.g., delete_case, delete_plan), making it somewhat vague in context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. The description lacks context about prerequisites (e.g., milestone existence), exclusions, or comparisons to related tools like update_milestone or get_milestone, leaving usage unclear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_planC

Delete a test plan

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full responsibility for behavioral disclosure. While 'Delete' clearly indicates a destructive operation, the description doesn't address critical behavioral aspects: whether deletion is permanent or reversible, what permissions are required, whether associated data (like plan entries or attachments) are also deleted, or what the response looks like. For a destructive tool with zero annotation coverage, this represents substantial missing context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is maximally concise at three words, with zero wasted language. It's appropriately sized for a simple destructive operation and gets straight to the point without unnecessary elaboration. The brevity is appropriate given the tool's straightforward purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive operation with no annotations and no output schema, the description is insufficiently complete. It doesn't address the behavioral implications of deletion, provide usage guidance relative to sibling tools, or explain what happens after invocation. Given the tool's potential impact and the rich ecosystem of related tools, more contextual information would help an AI agent use it appropriately and safely.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the single parameter 'plan_id' documented as 'Plan ID'. The description doesn't add any meaningful semantic context beyond what the schema already provides - it doesn't explain what constitutes a valid plan ID, where to find it, or format requirements. With complete schema coverage, the baseline score of 3 is appropriate as the schema does the parameter documentation work.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a test plan' clearly states the verb ('Delete') and resource ('test plan'), making the basic purpose understandable. However, it doesn't differentiate this from sibling tools like 'delete_case', 'delete_milestone', or 'delete_plan_entry' - all of which also delete specific resources in what appears to be a test management system. The description is functional but generic within this context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance about when to use this tool versus alternatives. There's no mention of prerequisites (e.g., whether the plan must be closed first), consequences (e.g., what happens to associated test cases or runs), or when not to use it. In a system with multiple deletion tools, this lack of differentiation is a significant gap for an AI agent trying to select the right tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_plan_entryC

Delete a test plan entry

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
entry_idYesPlan entry ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. 'Delete' implies a destructive mutation, but the description doesn't disclose whether this requires special permissions, whether deletion is permanent or reversible, what happens to associated data, or any rate limits. For a destructive operation with zero annotation coverage, this is a significant behavioral transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that states the core purpose without unnecessary words. It's appropriately sized for a simple deletion operation and front-loads the essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive mutation tool with no annotations and no output schema, the description is inadequate. It doesn't address behavioral aspects like permissions, permanence, side effects, or error conditions. Given the complexity of a deletion operation in what appears to be a test management system, more contextual information would be expected.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both parameters clearly documented in the schema. The description adds no additional parameter information beyond what's already in the structured schema. With complete schema coverage, the baseline score of 3 is appropriate as the description doesn't need to compensate for schema gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the target resource ('a test plan entry'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'delete_plan' or 'delete_case', but the verb+resource combination is specific enough for basic understanding.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With many sibling deletion tools (delete_plan, delete_case, delete_run, etc.), there's no indication of what distinguishes deleting a plan entry from deleting other entities or when this specific deletion operation is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_runC

Delete a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
softNoSet to 1 to preview deletion without executing (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. 'Delete' implies a destructive mutation, but the description doesn't disclose whether deletion is permanent, reversible, requires specific permissions, or has side effects (e.g., cascading deletions). The optional 'soft' parameter hints at preview capability, but this isn't explained in the description itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action and resource, making it immediately scannable. Every word earns its place in conveying the essential purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive tool with no annotations and no output schema, the description is inadequate. It doesn't explain what 'delete' entails (permanent vs. soft delete), what happens to associated data, error conditions, or return values. The agent lacks critical context needed to use this tool safely and effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no additional meaning about 'run_id' or 'soft' parameters beyond what's in the schema. This meets the baseline score of 3 when schema coverage is high, but doesn't compensate with extra context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a test run' clearly states the verb ('Delete') and resource ('a test run'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'delete_case', 'delete_plan', or 'delete_run_from_plan_entry', which all follow similar patterns but target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With many sibling deletion tools (e.g., delete_case, delete_plan, delete_run_from_plan_entry), there's no indication of which resource type this applies to or any prerequisites for deletion. The agent must infer usage from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_run_from_plan_entryC

Delete a test run from a plan entry

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the destructive action ('Delete') but doesn't mention permissions required, whether the deletion is permanent or reversible, or any side effects (e.g., impact on related data). This leaves significant gaps for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence with no wasted words. It's front-loaded with the core action and target, making it highly efficient and easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive tool with no annotations and no output schema, the description is insufficient. It lacks details on behavioral implications, error conditions, or what happens post-deletion. Given the complexity of mutation operations, more context is needed to ensure safe and correct usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description doesn't add any parameter details beyond the input schema, which has 100% coverage for the single parameter 'run_id'. Since the schema fully documents the parameter, the baseline score of 3 is appropriate, as no additional semantic value is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Delete') and the target ('a test run from a plan entry'), making the purpose immediately understandable. It distinguishes from sibling tools like 'delete_run' by specifying the context of a plan entry, though it doesn't explicitly contrast with all deletion alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'delete_run' or 'delete_plan_entry'. The description assumes the context is known but doesn't specify prerequisites, conditions, or exclusions for usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_sectionC

Delete a section

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesSection ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must convey behavioral traits. It only states the action (delete) without disclosing consequences (e.g., cascading deletes, permanence, required permissions). This is minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the purpose. It is appropriately sized, though it could include more detail without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity, the description is incomplete. A deletion tool should specify what happens to associated data, safety considerations, or return behavior to be fully informative.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% for the single parameter (section_id). The description adds no additional meaning beyond the schema's 'Section ID', so the value is at baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Delete a section' clearly states the verb (delete) and resource (section), making the purpose unambiguous. However, it does not differentiate from sibling tools like delete_case, but the name itself aids differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidelines are provided. There is no indication of when to use this tool versus alternatives (e.g., update_section) or any prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_attachmentC

Get/download an attachment by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
attachment_idYesAttachment ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'Get/download' which implies a read operation, but doesn't specify whether this requires authentication, what format the download returns (e.g., file content vs. metadata), or any rate limits. The description is minimal and lacks important behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise - a single phrase that communicates the core purpose without any wasted words. It's appropriately sized for a simple retrieval tool with one parameter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description is inadequate. It doesn't explain what format the attachment is returned in (e.g., binary data, download link, metadata), whether authentication is required, or how this differs from similar 'get_attachments_for_*' tools. The minimal description leaves too many questions unanswered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the single 'attachment_id' parameter. The description adds no additional semantic context about the parameter beyond what's in the schema (e.g., where to find attachment IDs, format requirements). Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get/download') and resource ('an attachment by ID'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'get_attachments_for_case' or 'delete_attachment' beyond the basic operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'get_attachments_for_case' or 'delete_attachment'. The description only states what it does, not when it's appropriate or what prerequisites might exist.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_attachments_for_caseC

Get all attachments for a test case

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesCase ID
limitNoLimit results (optional)
offsetNoOffset for pagination (optional)

TDQS

C2.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It implies a read operation ('Get'), but doesn't disclose behavioral traits such as permissions needed, rate limits, pagination behavior (beyond schema hints), or what 'attachments' include (e.g., file types, sizes). This is a significant gap for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words, making it easy to parse. However, it's front-loaded but could be more structured (e.g., by explicitly noting it's a list operation).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'attachments' entail (e.g., metadata, files), how results are returned, or error conditions. For a tool with 3 parameters and list functionality, more context is needed to guide the agent effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema documents all parameters (case_id, limit, offset). The description adds no meaning beyond the schema—it doesn't explain parameter relationships (e.g., limit/offset for pagination) or constraints. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the purpose ('Get all attachments for a test case') with a clear verb ('Get') and resource ('attachments'), but it's vague about scope ('all' is ambiguous without context) and doesn't distinguish from sibling tools like 'get_attachment' (singular) or 'get_attachments_for_plan' (different resource). It's adequate but lacks specificity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives is provided. The description doesn't mention sibling tools like 'get_attachment' (for a single attachment) or 'get_attachments_for_plan' (for attachments on plans), leaving the agent to infer usage from context alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_attachments_for_planC

Get all attachments for a test plan

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
limitNoLimit results (optional)
offsetNoOffset for pagination (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states the basic action. It doesn't disclose behavioral traits such as whether this is a read-only operation, potential rate limits, authentication needs, pagination behavior (implied by parameters but not explained), or what happens if the plan_id is invalid. This leaves significant gaps for a tool with parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and appropriately sized for its simple function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 3 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain return values (e.g., format of attachments list), error conditions, or behavioral context, leaving the agent with insufficient information for reliable use beyond basic parameter mapping.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so the schema already documents all parameters (plan_id, limit, offset) adequately. The description adds no additional meaning beyond implying 'all attachments' relates to plan_id, which is already clear from the schema. This meets the baseline of 3 for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and resource 'all attachments for a test plan', making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_attachments_for_case' or 'get_attachments_for_plan_entry', which would require a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'get_attachment' (for a single attachment) or other attachment-fetching siblings. It lacks any context about prerequisites, exclusions, or comparative use cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_attachments_for_plan_entryC

Get all attachments for a test plan entry

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
entry_idYesPlan entry ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Get') but doesn't clarify if this is a read-only operation, what permissions are required, whether it returns paginated results, or the format of the returned attachments. For a retrieval tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place without redundancy or unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a retrieval tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'attachments' entail (e.g., files, links), the return format, or error handling. For a tool that likely returns structured data, more context is needed to guide the agent effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with clear descriptions for both parameters (plan_id as a number, entry_id as a string). The description adds no additional meaning beyond the schema, such as explaining what a 'plan entry' is or providing examples. Given the high schema coverage, the baseline score of 3 is appropriate, as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Get') and resource ('all attachments for a test plan entry'), making the purpose specific and understandable. However, it doesn't explicitly distinguish this tool from sibling tools like 'get_attachments_for_case' or 'get_attachments_for_plan', which have similar naming patterns but target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid plan_id and entry_id), exclusions, or comparisons to sibling tools like 'get_attachment' (singular) or other attachment retrieval tools, leaving the agent to infer usage from naming alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_attachments_for_runB

Get all attachments for a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
limitNoLimit results (optional)
offsetNoOffset for pagination (optional)

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Get all attachments' but doesn't clarify if this is a read-only operation, what permissions are required, whether it's paginated (implied by limit/offset parameters but not stated), or what the return format looks like. For a tool with no annotation coverage, this leaves significant gaps in understanding its behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a straightforward retrieval tool, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's moderate complexity (retrieval with optional pagination), no annotations, and no output schema, the description is minimally adequate. It states what the tool does but lacks details on behavior, output format, or usage context. For a tool with 3 parameters and no structured safety hints, this leaves the agent with incomplete information to operate it effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents the three parameters (run_id, limit, offset). The description adds no additional meaning beyond the schema's parameter descriptions, such as explaining what 'attachments' include or how pagination works. With high schema coverage, the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get all attachments for a test run' clearly states the verb ('Get') and resource ('attachments for a test run'), making the purpose immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_attachments_for_case' or 'get_attachments_for_plan', which have similar patterns but target different resources.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a valid run_id), exclusions, or comparisons to siblings like 'get_attachment' (singular) or 'get_attachments_for_test', leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_attachments_for_testC

Get all attachments for a test

ParametersJSON Schema
NameRequiredDescriptionDefault
test_idYesTest ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It states the action ('Get') but lacks behavioral details: it doesn't specify if this is a read-only operation, what permissions are required, how results are returned (e.g., pagination, format), or error conditions. This is a significant gap for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It is front-loaded and appropriately sized for a simple tool, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description is incomplete. It lacks behavioral context (e.g., safety, permissions, return format) and doesn't compensate for the absence of structured data, making it inadequate for a tool that likely returns multiple attachments.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the single parameter 'test_id' documented in the schema as 'Test ID'. The description adds no additional parameter semantics beyond implying the tool uses a test ID, so it meets the baseline of 3 where the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get all attachments for a test' clearly states the verb ('Get') and resource ('attachments for a test'), making the purpose unambiguous. It distinguishes from siblings like 'get_attachment' (singular) and 'get_attachments_for_case' (different resource), though it doesn't explicitly contrast them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. While the description implies it's for retrieving attachments specifically for tests, it doesn't mention prerequisites (e.g., needing a valid test ID), exclusions, or compare it to similar tools like 'get_attachments_for_case' or 'get_attachment'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_caseB

Get a specific test case by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesCase ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose behavioral traits such as error behavior for non-existent IDs, authentication requirements, or whether the operation is read-only. Minimal information.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (one sentence) but lacks structure; it does not include any line breaks or additional formatting. It is under-specified for a complete tool description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has one parameter, no output schema, and no annotations, the description is minimal but technically sufficient for a simple get-by-ID operation. However, it could be improved by noting return type or error handling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already has 100% coverage by describing the case_id parameter. The description adds no further semantic meaning beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Get' and the resource 'a specific test case by ID', which is precise and distinguishes from sibling tools like get_cases (plural) or get_case_fields.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives (e.g., get_cases for listing). The description does not mention that a valid case_id is required or that the tool is only for individual retrieval.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_case_fieldsA

Get all available test case fields

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full responsibility for conveying behavior. It says 'Get all available test case fields' but does not explain what 'fields' means (e.g., schema definitions, values) or how the result is structured. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It is optimally concise for the simplicity of the tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no parameters and no output schema, the description is adequate but minimal. It explains the purpose but does not describe the output format or any prerequisites. Could be more complete given the lack of annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, and schema description coverage is 100% trivially. The description adds no parameter info but does not need to. Score is baseline 4 for zero-parameter tools.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and the resource ('all available test case fields'). It distinguishes this tool from siblings like get_case or get_cases, which retrieve case data rather than field definitions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like get_case_types or get_sections. With many sibling tools, explicit usage context is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_casesB

Get test cases for a project/suite

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
suite_idNoSuite ID (optional)
section_idNoSection ID (optional)

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided; description does not disclose read-only nature, error handling, or performance characteristics. Minimal behavioral context beyond implied retrieval.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single short sentence conveying core purpose. Could be slightly more structured but remains efficient with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Incomplete given no output schema and 3 parameters. Does not describe return format, pagination, or potential errors. Lacks details needed for robust usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all parameters. Description adds little beyond schema, but at baseline level as it does not contradict or enhance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it retrieves test cases for a project/suite, using specific verb and resource. Differentiates from siblings like 'get_case' (singular) and other retrieval tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like 'get_case' or 'get_results'. Lacks context for filtering or when to specify suite_id/section_id.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_case_statusesB

Get all available test case statuses (Enterprise)

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the action ('Get') but doesn't describe the return format (e.g., list of status objects, JSON structure), pagination behavior, error handling, or any rate limits. For a read operation with zero annotation coverage, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It front-loads the key information ('Get all available test case statuses') and includes a brief qualifier '(Enterprise)' for added context, making it easy to scan and understand quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (0 parameters, no output schema, no annotations), the description is adequate but has clear gaps. It lacks details on the return format, which is critical since there's no output schema, and doesn't address behavioral aspects like error handling. For a read-only tool with no structured output documentation, more completeness would be beneficial.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0 parameters with 100% coverage, so the schema fully documents that no inputs are required. The description doesn't add parameter details, which is appropriate here, but it could have mentioned if any implicit parameters (like authentication context) are needed. Baseline is 4 for 0 parameters, as the description doesn't need to compensate for gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Get') and resource ('all available test case statuses'), with the parenthetical '(Enterprise)' providing additional context about the scope or version. However, it doesn't explicitly differentiate from the sibling tool 'get_statuses' (which appears to be a similar but potentially different tool), leaving some ambiguity about when to use one versus the other.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'get_statuses' or other status-related tools. It lacks context about prerequisites, such as whether it requires authentication or specific project access, and doesn't mention any exclusions or limitations in usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_case_typesB

Get all available test case types

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so the description carries full responsibility. It does not disclose any behavioral traits such as idempotency, authentication requirements, or effects. The description is minimal and provides no insight into side effects or state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short (one sentence) with no wasted words, appropriate for a zero-parameter tool. However, it could be slightly more informative without compromising conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters and no output schema, the description is functionally adequate but lacks context about what the returned data represents (e.g., IDs, names). It does not mention any prerequisites or related operations, leaving room for ambiguity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (no parameters), so the description adds no additional meaning beyond the schema. Baseline is 3, and the description simply restates the resource name without enriching parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and the resource ('all available test case types'). It distinguishes itself from siblings like get_case_fields or get_cases by specifying 'case types' as a distinct entity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool vs alternatives, but the action is straightforward (listing types) and likely used before creation operations. The description implies usage context but lacks detailed when-to-use instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_configsC

Get all configurations for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states a read operation ('Get'), implying it is likely safe and non-destructive, but does not specify details like authentication needs, rate limits, error handling, or the format/scope of returned configurations (e.g., whether it returns all configurations at once or supports pagination). This leaves significant gaps for a tool with no annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no wasted words, making it easy to parse and front-loaded with the core purpose. It efficiently conveys the essential information without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete. It does not address behavioral aspects like response format, error conditions, or system constraints, which are critical for a read operation tool. While concise, it fails to provide sufficient context for reliable agent use beyond the basic purpose.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with the single parameter 'project_id' clearly documented. The description adds no additional semantic context beyond implying the parameter is used to scope configurations to a specific project, which is already evident from the schema. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('configurations for a project'), making the purpose immediately understandable. However, it does not distinguish this tool from its many sibling 'get_' tools (e.g., get_case, get_plan, get_results), which all follow a similar pattern of retrieving specific data, so it lacks explicit differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With many sibling tools available (e.g., get_project, get_cases), there is no indication of context, prerequisites, or exclusions, leaving the agent to infer usage based on the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_current_userB

Get the current authenticated user

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, the description doesn't specify what information is returned, authentication requirements, error conditions, or rate limits. It mentions 'authenticated user' but doesn't clarify what happens if no user is authenticated.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that communicates the core purpose without any wasted words. It's appropriately sized for a simple tool with no parameters and gets straight to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a user retrieval tool with no annotations and no output schema, the description is insufficient. It doesn't explain what user information is returned (e.g., ID, name, email, permissions), authentication requirements, or error handling. The agent would need to guess about the response format and behavioral characteristics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0 parameters and 100% schema description coverage, the baseline is 4. The description appropriately doesn't discuss parameters since none exist, and the schema already fully documents this. No additional parameter information is needed or provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('current authenticated user'), making the purpose immediately understandable. It doesn't explicitly distinguish from sibling tools like 'get_user' or 'get_user_by_email', but the focus on 'current authenticated' user provides some implicit differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'get_user' or 'get_user_by_email'. It doesn't mention prerequisites (e.g., authentication status) or specific contexts where this tool is appropriate versus other user retrieval methods.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_history_for_caseC

Get the edit history for a test case

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesCase ID
limitNoLimit results (optional)
offsetNoOffset for pagination (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It states it 'gets' data (implying read-only), but doesn't disclose behavioral traits like authentication requirements, rate limits, error conditions, or what 'edit history' includes (e.g., timestamps, users, changes). For a tool with no annotation coverage, this is insufficient transparency about how it behaves.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, clear sentence with zero waste. It's appropriately sized for a simple retrieval tool and front-loads the core purpose. Every word earns its place without redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description is incomplete. It doesn't explain what 'edit history' returns (e.g., list of changes, timestamps, users), pagination behavior (implied by offset/limit but not described), or error handling. For a tool with 3 parameters and rich sibling context, more completeness is needed to guide the agent effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so parameters are fully documented in the schema. The description adds no additional meaning about parameters beyond implying 'case_id' identifies the test case. With high schema coverage, the baseline is 3 even without parameter details in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Get') and resource ('edit history for a test case'), making the purpose understandable. It doesn't explicitly differentiate from siblings like 'get_case' or 'get_results_for_case', but the focus on 'edit history' provides reasonable distinction. A 5 would require explicit sibling comparison.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. With many sibling tools for retrieving case-related data (e.g., 'get_case', 'get_results_for_case', 'get_attachments_for_case'), the description offers no context about when edit history is needed versus other case information. This leaves the agent without usage direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_milestoneA

Get a specific milestone by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
milestone_idYesMilestone ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. Description indicates a read operation but does not disclose error handling, permissions, or what happens if the milestone is not found.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no wasted words, easy to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get-by-ID tool with one parameter and no output schema, the description is mostly complete but could mention the return format.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already explains the parameter. Description adds no extra meaning beyond 'Milestone ID'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the verb 'Get' and resource 'a specific milestone by ID', distinguishing it from the sibling tool 'get_milestones'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like 'get_milestones'. Implied by naming but not stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_milestonesB

Get milestones for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fails to disclose behavioral details such as whether the operation is read-only, idempotent, or requires specific permissions. It does not describe the return type or any side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely brief but lacks structure or elaboration. While concise, it borders on under-specification.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (one parameter, no output schema, no annotations), the description minimally covers the purpose. However, it could be more complete by indicating that it returns a list of milestones or the expected output.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with the single parameter 'project_id' described as 'Project ID'. The tool description adds no extra semantic value beyond the schema, fitting the baseline of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses the specific verb 'Get' and clearly identifies the resource 'milestones for a project'. It distinguishes itself from the sibling 'get_milestone' which retrieves a single milestone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'get_milestone'. No prerequisites or context for usage are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_planC

Get a specific test plan by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. The description only states 'Get a specific test plan by ID', implying a read-only operation, but does not disclose behavior on missing IDs, error handling, or any side effects. The description lacks needed transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise, consisting of a single phrase. It contains no fluff but is perhaps too brief; it could benefit from additional context without sacrificing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, no output schema, and the sibling toolset, the description is minimally adequate. It covers the core action but omits return value context or any edge cases, making it moderately complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter plan_id, which is described as 'Plan ID'. The tool description adds 'by ID' reinforcing the parameter purpose but does not add new meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get a specific test plan by ID' clearly states the verb and resource, distinguishing from sibling tools like get_plans which lists plans. However, it does not explicitly differentiate from similar tools like get_case or get_suite, but the resource is specific enough.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like get_plans for listing. The description implies usage for a single plan but does not mention prerequisites or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_plansC

Get test plans for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full responsibility for behavioral disclosure. It only states a generic read operation ('Get') without mentioning any side effects, rate limits, pagination, or filtering behavior. The input schema covers project_id, but the description omits any additional behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short (one sentence) and front-loaded, but this brevity sacrifices informativeness. While it avoids wasted words, it does not earn its place by providing sufficient detail beyond the tool's name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of an output schema, the description should clarify what is returned (e.g., a list of test plans, details). It does not. The tool has 1 parameter and is simple, but the description fails to cover the return type or any edge cases, leaving the agent with incomplete context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single parameter 'project_id'. The description does not add any meaning beyond the schema's 'Project ID' label. The baseline score of 3 is appropriate since the schema already documents the parameter adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('test plans') with scope ('for a project'). It is not a tautology and distinguishes from similar verb-noun combinations. However, it does not differentiate from the sibling tool 'get_plan' (singular), which could confuse an agent deciding between the two.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'get_plan', 'get_cases', or 'get_sections'. The agent receives no context about the expected output or prerequisites, leaving it to infer usage solely from the name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prioritiesB

Get all available test case priorities

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It states 'Get all available test case priorities', implying a read-only operation, but doesn't disclose behavioral traits like whether it requires authentication, returns a list or object, has pagination, or potential errors. This is inadequate for a tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly. Every word earns its place without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 0 parameters, 100% schema coverage, and no output schema, the description is minimally complete but lacks depth. It doesn't explain return values (e.g., format, structure) or behavioral context, which is a gap since annotations are absent. For a simple read tool, it's adequate but could be more informative.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters with 100% schema description coverage, so no parameter information is needed. The description doesn't add param details, which is fine given the schema fully covers it. A baseline of 4 is appropriate as the description doesn't need to compensate for any gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Get') and resource ('all available test case priorities'), making the purpose specific and understandable. However, it doesn't differentiate from sibling tools like 'get_case_statuses' or 'get_case_types' that also retrieve metadata, leaving room for confusion about when to use this versus similar 'get_' tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. With many sibling tools (e.g., 'get_case_statuses', 'get_case_types'), the description lacks context on prerequisites, typical use cases, or exclusions, leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_projectB

Get a specific project by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided. Description only says 'get', implying read-only, but fails to disclose return format, permissions, or error behavior. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence is efficient and front-loaded. However, it is too brief to add value beyond the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, no annotations, and no behavioral details. A simple fetch tool should mention what it returns or any side effects. Incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters with description 'Project ID'. Description adds no additional meaning beyond what schema already provides. Baseline 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get'), resource ('project'), and method ('by ID'). It effectively distinguishes from sibling tools like 'get_projects'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this vs alternatives like 'get_suite' or 'get_case'. No prerequisites or exclusions mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_projectsB

Get all TestRail projects

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description bears full burden. It does not disclose whether the operation is read-only, requires authentication, or handles potential large result sets (e.g., pagination). The description is minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, short sentence that is front-loaded and concise. Every word adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with no parameters and no output schema, the description adequately communicates the purpose. However, it could mention that the tool returns a list of projects or the format of the output, slightly improving completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are no parameters, and schema coverage is 100% (empty schema). Baseline is 3 per instructions. The description adds no additional parameter information beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get all TestRail projects' clearly states the action ('Get') and the resource ('all TestRail projects'), distinguishing it from siblings like 'get_project' which retrieves a single project.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. For example, if a user needs a single project, they should use 'get_project' instead. Missing context on when 'all' is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_result_fieldsB

Get all available result custom fields

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states the basic action without disclosing behavioral traits. It doesn't mention whether this is a read-only operation, if it requires authentication, potential rate limits, or the format of the returned data (e.g., list of field objects). This leaves significant gaps for an agent to understand how to handle the tool effectively.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and wastes no space, making it easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (a read operation with no parameters) and lack of annotations and output schema, the description is incomplete. It doesn't explain what 'result custom fields' entail (e.g., types, usage) or the return format, leaving the agent without enough context to use the tool effectively beyond the basic action.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate here, earning a baseline score of 4 as it doesn't need to compensate for any schema gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('all available result custom fields'), providing a specific purpose. However, it doesn't explicitly differentiate from sibling tools like 'get_case_fields' or 'get_statuses', which might retrieve similar metadata for other entities.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. The description lacks context about prerequisites (e.g., needing a project or result ID) or comparisons to other metadata-fetching tools in the sibling list, such as 'get_case_fields' or 'get_priorities'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_resultsC

Get results for a test

ParametersJSON Schema
NameRequiredDescriptionDefault
test_idYesTest ID

TDQS

C2.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states the basic action. It fails to disclose behavioral traits like read-only nature, authentication needs, or response format.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short (5 words), which is efficient but lacks necessary detail. It is front-loaded but does not earn its place due to missing context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the single parameter, no output schema, no annotations, and many similar sibling tools, the description is minimally complete. It does not clarify the scope of results or relationship to other tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% (one parameter with basic description 'Test ID'), so baseline is 3. The tool description adds no additional meaning to the parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states 'Get results for a test', which is a clear verb+noun combination, but it does not differentiate from sibling tools like get_results_for_case and get_results_for_run, leaving ambiguity about scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives. The description provides no contextual cues, such as when to choose get_results over more specific result retrieval tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_results_for_caseB

Get results for a test case in a run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
case_idYesCase ID

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It states the operation is a retrieval (read), which implies a safe, non-destructive action. However, it does not disclose any behavioral traits such as pagination, ordering, filtering, or whether results are for all runs or specific ones. For a simple read operation, this is adequate but could be improved.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is to the point and front-loaded. It uses no unnecessary words. While brief, it could benefit from additional context, but it is not verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has only two parameters and no output schema, the description is somewhat complete. However, it lacks details on the return value format or what exactly constitutes 'results'. For a retrieval tool, this information would help the agent use the output correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers both parameters with descriptions ('Run ID', 'Case ID'), achieving 100% coverage. The description adds no additional meaning beyond what the schema already provides, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get results for a test case in a run' uses a specific verb ('Get') and clearly identifies the resource ('results for a test case in a run'). It distinguishes itself from sibling tools like 'get_results_for_run' (which gets results for an entire run) and 'get_case' (which gets case details).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide any guidance on when to use this tool versus alternatives. It lacks explicit context for when this tool should be preferred over siblings like 'get_results_for_run' or 'get_case', and does not mention any prerequisites or limitations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_results_for_runC

Get results for a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so the description should disclose behavior. It only says 'Get results' implying read-only, but omits details like response format, pagination, or what constitutes a result.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is concise and front-loaded. However, it is too brief and does not fully earn its place due to lack of value-added detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description is minimally adequate. It states the purpose but lacks complete guidance on return values or behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with the parameter described as 'Run ID'. The description adds no additional meaning beyond what the schema provides, warranting the baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource, clearly stating it retrieves results for a test run. It distinguishes from siblings like 'get_results_for_case' but could be more explicit about the scope compared to 'get_results'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like 'get_results' or 'get_results_for_case'. The description does not mention context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_runB

Get a specific test run by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the burden. It only repeats the function implied by the name, without disclosing behavioral details such as whether the run is returned, its structure, or error states.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence with no wasted words, but it could be slightly more informative while remaining concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get-by-ID tool, the description is adequate but lacks information about return values, given no output schema. It does not cover potential errors or limitations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% (one parameter with description). The description adds no further meaning beyond the schema, so baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and the resource 'a specific test run by ID', distinguishing it from sibling tools like get_runs (plural) and other get_* tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives such as get_runs, get_tests, etc. The description does not mention context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_runsB

Get test runs for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description adds no behavioral details beyond the basic operation. It does not disclose if there are pagination, ordering, or filtering capabilities, nor any side effects or performance implications.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence that is front-loaded and contains no extraneous information. It is as concise as possible while conveying the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one parameter and no output schema, the description is minimally adequate. However, it lacks context about what test runs are or how they relate to other entities, which could be helpful for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of the parameter (project_id) with a description. The description adds no additional meaning beyond the schema, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get test runs for a project' clearly states the verb (Get) and resource (test runs) with a scope (for a project). However, it does not differentiate from sibling tools like 'get_run' or 'get_tests', which could confuse an agent.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. For instance, it does not mention that 'get_run' is for a single run or that this lists all runs, missing an opportunity to clarify selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_sectionA

Get a specific section by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesSection ID

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose behavioral traits such as required permissions, potential errors, or data scope. It only states the basic action.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence is concise and front-loads the purpose. However, it could include slightly more context without being verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get-by-ID tool with one parameter and no output schema, the description is complete enough. It provides the essential information needed for correct tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with a description for 'section_id'. The tool description adds no additional meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get a specific section by ID' uses a specific verb and resource, clearly identifying the tool's action. It distinguishes well from sibling tools like 'get_sections' (list) and 'add_section' (create).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like 'get_sections'. Usage is implied from the description but not clarified.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_sectionsB

Get all sections for a project/suite

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID
suite_idNoSuite ID (optional)

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. Does not disclose behavioral traits such as idempotency, side effects, authentication requirements, or what happens if no sections are found. Only implies a read operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence achieving maximum conciseness. Front-loaded with core purpose, no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list retrieval tool, description is adequate. However, no output schema means an agent might benefit from knowing return format. Given the many siblings, a note about relationship to get_section or get_suite would improve completeness. Slightly above baseline due to clarity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions are minimal (Project ID, Suite ID optional). Description adds context that suite_id filters sections within a project, but schema coverage is 100%, so baseline is 3. No additional meaningful detail beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get all sections for a project/suite' clearly states the verb (Get) and resource (sections) with scope. It distinguishes from sibling 'get_section' which retrieves a single section, and other get_* tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool vs alternatives. Does not mention that 'get_section' should be used for a specific section, or provide context for choosing over other list tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_statusesB

Get all available test result statuses

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. It only states what the tool does ('Get all available test result statuses') without mentioning permissions, rate limits, response format, or any side effects. This leaves significant gaps for a tool with zero annotation coverage.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without any unnecessary words. It's appropriately sized and front-loaded, making it easy to understand at a glance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'statuses' entail, the return format, or any behavioral context needed for proper use. For a tool in a complex system with many siblings, more detail would be beneficial.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters with 100% schema description coverage, meaning the schema fully documents the lack of inputs. The description doesn't need to add parameter information, so it meets the baseline of 4 for tools with no parameters, as it doesn't contradict or omit anything.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('all available test result statuses'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'get_case_statuses' or other status-related tools, which would require explicit distinction to earn a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With siblings like 'get_case_statuses' and 'get_results' that might overlap in context, there's no explicit or implied direction on appropriate usage scenarios or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_suiteA

Get a specific test suite by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
suite_idYesSuite ID

TDQS

A3.6/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must convey behavior fully. It only says 'Get', implying read-only, but does not disclose what happens for invalid IDs (error response), whether the output is the full suite object, or any side effects. This is insufficient for complete transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence that immediately states the core purpose. There is no extraneous information; every word is necessary.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity (1 required param, no output schema, no annotations, simple resource), the description is mostly complete. It covers what the tool does and the key parameter. The lack of usage guidance is a minor gap, but the description suffices for a straightforward retrieval tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with one parameter 'suite_id' described as 'Suite ID'. The description adds 'by ID' but no additional semantics, format, or constraints beyond the schema. Baseline 3 is appropriate since the schema handles the param documentation adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states 'Get a specific test suite by ID,' clearly identifying the action (get), resource (test suite), and scope (by ID). It naturally distinguishes from sibling tools like 'get_suites' (lists all) and 'add_suite' (creates).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for retrieving a single test suite by ID, but provides no explicit guidance on when to use this tool versus alternatives, nor any prerequisites or conditions (e.g., 'use when you have the suite_id'). Minimal but adequate for a simple tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_suitesB

Get all test suites for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, and description fails to disclose behavioral traits such as pagination, ordering, or error handling, which are important for a list operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Single sentence, no extraneous words. However, could be slightly expanded to cover behavioral context without losing conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple list tool with one parameter and no output schema, description minimally covers purpose, but lacks details on return format, potential limits, or error cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with one parameter 'project_id' described succinctly. Description adds no extra meaning beyond the schema, placing it at baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Get all test suites for a project', using a specific verb and resource, and distinguishes from sibling tools like 'get_suite' (single suite) and 'add_suite'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Description implies usage for listing all suites, but lacks explicit guidance on when to use this vs alternatives like 'get_suite' or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_templatesC

Get all templates for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states 'Get all templates' but does not clarify if this is a read-only operation, what permissions are required, whether results are paginated, or if it returns an error for invalid project IDs. For a retrieval tool with zero annotation coverage, this is a significant gap in transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no wasted words. It is front-loaded with the core action and resource, making it easy to parse. Every word contributes directly to understanding the tool's purpose, achieving optimal conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema, no annotations), the description is incomplete. It lacks details on behavioral aspects like error handling or result format, and does not guide usage relative to siblings. While concise, it fails to provide sufficient context for an agent to use the tool effectively without additional inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage, with 'project_id' documented as 'Project ID'. The description adds minimal value by implying the parameter is used to scope templates to a project, but does not provide additional context like format examples or constraints. With high schema coverage, the baseline score of 3 is appropriate as the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get all templates for a project' clearly states the action (get) and resource (templates), with context (for a project). It distinguishes from siblings like 'get_sections' or 'get_suites' by specifying templates, but does not explicitly differentiate from tools like 'get_case_fields' or 'get_configs' that might also retrieve project-related data. The purpose is specific but lacks explicit sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, such as needing a valid project ID, or exclusions, like whether it retrieves only active templates. With many sibling tools for retrieving various project entities, this lack of context leaves the agent to infer usage based on the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_testB

Get a specific test by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
test_idYesTest ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, and the description does not disclose behavioral details such as whether the operation is read-only, what happens if the test_id does not exist, or any side effects. The description simply states the basic retrieval function.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no unnecessary words. It is perfectly concise and front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (one parameter, no output schema), the description is minimally adequate but lacks details about the return format, error cases, or behavior when multiple tests might match.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of parameters, so the baseline is 3. The description adds no additional meaning beyond what the schema provides (the param is described as 'Test ID' in the schema, and the description just says 'by ID').

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (get) and the resource (a specific test) and the method (by ID). It distinguishes from sibling tools like get_tests, get_case, etc., by being specific to a single test resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives like get_tests (which retrieves multiple tests) or other get_* tools. There is no mention of prerequisites or context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_testsC

Get tests for a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, and the description does not disclose behavioral traits such as read-only nature, pagination, sorting, or any side effects. It only states the basic function without clarifying what 'tests' entails or how the data is returned. The description carries the full burden for transparency but falls short.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very concise, consisting of a single sentence with no redundant words. It is front-loaded with the key information. However, it could benefit from slightly more structure, such as listing parameters or return value, while still being concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool (one required parameter, no output schema), the description is minimally complete. It explains what the tool does and what input is needed, but it omits details about the output format, possible empty results, or error conditions. For a tool of this complexity, it is adequate but not thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema coverage is 100% with one parameter 'run_id' described as 'Run ID'. The description adds context by stating 'for a test run', which reinforces the parameter's purpose. However, it does not provide additional semantic detail beyond what the schema already provides, so it meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get tests for a test run' clearly states the verb ('get') and resource ('tests'), and relates it to a test run. It is specific enough to distinguish from sibling tools like 'get_test' (singular) by implying a collection, but it does not explicitly differentiate from similar tools like 'get_results_for_run'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidelines are provided. The description does not indicate when to use this tool versus alternatives like 'get_test' for a single test or 'get_results_for_run' for results. There are no explicit when-to-use or when-not-to-use instructions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_userC

Get a user by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesUser ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but offers minimal behavioral insight. It doesn't disclose whether this is a read-only operation, what happens if the user ID doesn't exist, authentication needs, rate limits, or response format. 'Get' implies retrieval but lacks operational details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with a single, clear sentence that front-loads the essential information. There's zero wasted language, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no annotations and no output schema, the description is incomplete. It doesn't explain what data is returned, error conditions, or behavioral traits. While simple, it leaves the agent guessing about the operation's full context and results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% with the parameter 'user_id' documented as 'User ID'. The description adds no additional meaning beyond what the schema provides, such as format examples or constraints. Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and resource ('a user by ID'), making the purpose immediately understandable. It distinguishes from siblings like 'get_users' (plural) by specifying singular retrieval, but doesn't explicitly differentiate from 'get_user_by_email' which serves a similar purpose with a different identifier.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. It doesn't mention when to choose 'get_user' over 'get_user_by_email' or 'get_users', nor does it specify prerequisites like authentication requirements or access permissions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_by_emailB

Get a user by email address

ParametersJSON Schema
NameRequiredDescriptionDefault
emailYesUser email address

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It states the tool's function but doesn't disclose behavioral traits like whether it's read-only (implied by 'Get'), what happens if the email isn't found (returns null/error), authentication needs, rate limits, or response format. For a lookup tool with zero annotation coverage, this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a simple lookup tool. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's low complexity (single parameter, no output schema, no annotations), the description is minimally adequate but lacks completeness. It doesn't cover behavioral aspects like error handling or response format, which are important for a lookup tool. With no annotations or output schema, more context would be helpful.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the parameter 'email' fully documented in the schema. The description adds no additional parameter semantics beyond what's in the schema (e.g., email format, validation rules). Baseline 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get') and target resource ('a user'), specifying the lookup method ('by email address'). It distinguishes from sibling 'get_user' (which likely uses ID) and 'get_users' (which lists multiple users), but doesn't explicitly mention these alternatives. The purpose is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage context (when you need to find a user by email rather than ID), but doesn't explicitly state when to use this vs 'get_user' or 'get_users'. No guidance on prerequisites, error conditions, or alternatives is provided. Usage is clear from the name but not elaborated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_usersC

Get all users (optionally filtered by project)

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idNoProject ID (optional, required for non-admins)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It mentions optional filtering but doesn't disclose behavioral traits like pagination, rate limits, authentication requirements, or what 'all users' means in scope (e.g., active only, all roles). For a read operation with no annotations, this leaves significant gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core purpose ('Get all users') and adds optional detail. There is no wasted text, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no annotations and no output schema, the description is incomplete for a tool that likely returns a list of users. It doesn't explain return format, pagination, error handling, or permissions (e.g., admin vs. non-admin access implied by parameter note but not clarified). For a read tool with potential complexity, this is inadequate.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with the parameter 'project_id' documented in the schema. The description adds minimal value by implying filtering functionality but doesn't provide additional context like format examples or edge cases. Baseline 3 is appropriate given high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Get' and resource 'users', with optional filtering by project. It distinguishes from sibling 'get_user' (singular) but not explicitly from 'get_user_by_email'. The purpose is specific but could better differentiate from similar tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like 'get_user' or 'get_user_by_email'. The description mentions optional filtering but doesn't explain when filtering is appropriate or required. No prerequisites or exclusions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

move_cases_to_sectionC

Move test cases to another section

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesTarget section ID
suite_idYesTarget suite ID
case_idsYesArray of case IDs to move

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden but only states the action without behavioral details. It doesn't disclose if this is destructive (e.g., removes cases from original location), requires specific permissions, has side effects, or handles errors, leaving critical gaps for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words, front-loading the core action. It's appropriately sized for the tool's complexity, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks behavioral context, error handling, or result details, failing to compensate for the absence of structured data, which could lead to misuse by an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema fully documents parameters like 'section_id' and 'case_ids'. The description adds no additional meaning beyond implying movement involves these parameters, meeting the baseline for high schema coverage without extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('move') and resources ('test cases to another section'), making the purpose immediately understandable. It doesn't differentiate from sibling tools like 'copy_cases_to_section' or 'move_section', but it's not vague or tautological.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'copy_cases_to_section' or 'move_section'. It lacks context about prerequisites, such as whether cases must exist or sections be accessible, leaving the agent without usage direction.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

move_sectionC

Move a section to another parent or position

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesSection ID
parent_idNoParent section ID (can be null for root)
after_idNoSection ID after which to place this section (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('move') but doesn't clarify if this requires specific permissions, whether it's destructive to data, what happens to child elements, or error conditions. This leaves significant gaps for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded and wastes no space, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is incomplete. It lacks details on behavioral traits, error handling, return values, and usage context, leaving the agent with insufficient information to invoke it safely and effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all three parameters. The description adds minimal value by implying the parameters relate to parent/position changes but doesn't provide additional syntax, constraints, or examples beyond the schema's basic descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('move') and resource ('section'), specifying the action of relocating a section to another parent or position. However, it doesn't explicitly differentiate from sibling tools like 'move_cases_to_section', which handles different resources, leaving some ambiguity about scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, such as 'update_section' for other modifications or 'move_cases_to_section' for different operations. It lacks context about prerequisites, dependencies, or typical scenarios for moving sections.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_caseC

Update an existing test case

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesCase ID
titleNoTest case title (optional)
template_idNoTemplate ID (optional)
type_idNoTest case type ID (optional)
priority_idNoPriority ID (optional)
estimateNoTime estimate (optional)
refsNoReferences (optional)
custom_precondsNoPreconditions (optional)
custom_stepsNoTest steps (optional)
custom_expectedNoExpected result (optional)
custom_steps_separatedNoSeparated test steps (optional). Array of step objects with content/expected or shared_step_id

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. 'Update an existing test case' implies a mutation operation but reveals nothing about permissions required, whether changes are reversible, error conditions, or what happens to unspecified fields (partial vs full updates). For a mutation tool with 11 parameters, this leaves critical behavioral aspects undocumented.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that states the core purpose without unnecessary words. It's perfectly front-loaded and wastes no space on redundant information. Every word earns its place in this minimal but complete statement of function.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 11 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't address critical context like authentication requirements, error handling, what constitutes a successful update, or the response format. The agent lacks necessary information to use this tool effectively beyond basic parameter passing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, so all parameters are documented in the schema itself. The description adds no additional parameter semantics beyond what's already in the schema - it doesn't explain relationships between parameters, formatting requirements, or business rules. This meets the baseline for high schema coverage but adds no extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Update') and resource ('an existing test case'), making the purpose immediately understandable. It distinguishes from sibling tools like 'add_case' by specifying 'existing', but doesn't explicitly differentiate from other update tools like 'update_milestone' or 'update_plan' beyond the resource type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (like needing a valid case_id), when to choose this over 'add_case', or how it relates to other update operations in the system. The agent must infer usage from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_milestoneC

Update an existing milestone

ParametersJSON Schema
NameRequiredDescriptionDefault
milestone_idYesMilestone ID
nameNoMilestone name (optional)
descriptionNoMilestone description (optional)
due_onNoDue date as UNIX timestamp (optional)
is_completedNoMark as completed (optional)
is_startedNoMark as started (optional)
parent_idNoParent milestone ID (optional)
start_onNoStart date as UNIX timestamp (optional)

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. 'Update an existing milestone' implies a mutation operation but reveals nothing about permissions needed, whether changes are reversible, rate limits, error conditions, or what happens to unspecified fields. For a mutation tool with 8 parameters, this is a significant gap in behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with good schema documentation and gets straight to the point without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 8 parameters and no annotations or output schema, the description is inadequate. It doesn't explain what 'update' entails operationally, what the response looks like, error handling, or how it interacts with the system. The agent lacks crucial context for proper tool invocation despite good schema coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with all 8 parameters well-documented in the input schema. The description adds no parameter information beyond what's already in the schema, so it meets the baseline of 3 where the schema does the heavy lifting. No additional semantic context is provided.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update an existing milestone' clearly states the action (update) and resource (milestone), but it's vague about what aspects can be updated and doesn't distinguish this tool from sibling update tools like update_case or update_plan. It provides basic purpose but lacks specificity about scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites (e.g., needing an existing milestone), when not to use it, or how it differs from other update tools in the sibling list. The agent must infer usage from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_planC

Update an existing test plan

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
nameNoPlan name (optional)
descriptionNoPlan description (optional)
milestone_idNoMilestone ID (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden for behavioral disclosure. It states this is an update operation (implying mutation), but doesn't cover critical aspects like required permissions, whether changes are reversible, error conditions (e.g., invalid plan_id), or what happens to unspecified fields. This leaves significant gaps for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core action ('Update an existing test plan'), making it immediately scannable and appropriately sized for its purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, the description is incomplete. It doesn't address behavioral aspects (permissions, side effects), response format, or error handling. While the schema covers parameters well, the overall context for safe and effective use is insufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are documented in the schema. The description doesn't add any meaning beyond what's in the schema (e.g., it doesn't explain relationships between parameters or provide examples). The baseline score of 3 reflects adequate parameter documentation through the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Update') and resource ('an existing test plan'), making the purpose immediately understandable. It distinguishes from sibling tools like 'add_plan' (create) and 'delete_plan' (remove), but doesn't specify what aspects can be updated beyond the generic term 'update'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. The description doesn't mention prerequisites (e.g., needing an existing plan ID), exclusions, or comparisons to similar tools like 'update_plan_entry' or 'update_case'. The agent must infer usage from the name and schema alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_plan_entryC

Update a test plan entry

ParametersJSON Schema
NameRequiredDescriptionDefault
plan_idYesPlan ID
entry_idYesPlan entry ID
nameNoEntry name (optional)
descriptionNoEntry description (optional)
assignedto_idNoUser ID to assign (optional)
include_allNoInclude all test cases (optional)
case_idsNoSpecific case IDs (optional)
refsNoReferences (optional)

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. 'Update a test plan entry' implies a mutation operation but provides no information about permissions required, whether changes are reversible, what happens to unspecified fields, error conditions, or response format. For a mutation tool with 8 parameters and no annotation coverage, this leaves significant behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a tool with good schema documentation and gets straight to the point without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 8 parameters, no annotations, and no output schema, the description is insufficiently complete. It doesn't address behavioral aspects like permissions, side effects, or response format, nor does it differentiate from similar sibling tools. The agent would need to make significant assumptions about how to properly use this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description coverage is 100%, with all 8 parameters clearly documented in the input schema. The description adds no additional parameter information beyond what's already in the schema. According to the scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update a test plan entry' clearly states the action (update) and resource (test plan entry), which is specific enough to understand the basic function. However, it doesn't differentiate from sibling tools like 'update_plan' or 'update_run_in_plan_entry', leaving ambiguity about what specifically distinguishes this update operation from other update operations in the system.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. With multiple sibling update tools (update_case, update_milestone, update_plan, update_run, update_run_in_plan_entry, update_section, update_suite), there's no indication of what makes this tool distinct or when it should be selected over other update operations. The agent must infer usage from the tool name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_runC

Update an existing test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
nameNoRun name (optional)
descriptionNoRun description (optional)
milestone_idNoMilestone ID (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the full burden. It only states the basic verb (update) without disclosing behavior like what happens if the run_id does not exist, whether the update is partial or full replacement, or any side effects like triggering notifications.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that front-loads the purpose. It is appropriate in length but could include slightly more context without becoming verbose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has four parameters and no output schema, the description is incomplete. It does not mention what the response looks like, error conditions, or that the run must exist. A mutation tool like this requires more behavioral context for safe use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds no additional meaning about parameters beyond the schema. It does not explain, for example, how milestone_id relates to the run or constraints like uniqueness.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Update an existing test run' clearly states the action and resource, distinguishing it from sibling tools like 'add_run' (create) and 'delete_case' (delete). However, it does not explicitly specify what aspects of a run can be updated, which is partially covered by the input schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, such as requiring a valid run_id or that updates are partial, nor any indication of when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_run_in_plan_entryC

Update a test run inside a plan entry

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesRun ID
descriptionNoRun description (optional)
assignedto_idNoUser ID to assign (optional)
include_allNoInclude all test cases (optional)
case_idsNoSpecific case IDs (optional)
refsNoReferences (optional)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. 'Update' implies mutation, but the description doesn't disclose behavioral traits like whether this requires specific permissions, what happens to existing run data not mentioned, whether changes are reversible, or any rate limits. It only states the basic action without operational context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence that front-loads the core action ('Update a test run') and context ('inside a plan entry') with zero wasted words. Every word earns its place in conveying the essential information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with 6 parameters, no annotations, and no output schema, the description is insufficient. It doesn't address what the tool returns, error conditions, or behavioral implications of updating a run within a plan entry versus standalone. The context signals indicate complexity that isn't matched by descriptive completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are documented in the schema. The description adds no additional meaning about parameters beyond what's in the schema (e.g., it doesn't explain relationships between 'include_all' and 'case_ids' or what 'refs' format should be). Baseline 3 is appropriate when schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Update') and resource ('a test run inside a plan entry'), making the purpose immediately understandable. It distinguishes this from generic 'update_run' by specifying the 'inside a plan entry' context, though it doesn't explicitly differentiate from sibling 'update_run' which might update runs outside plan entries.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives like 'update_run' or 'update_plan_entry'. It mentions the context ('inside a plan entry') but doesn't explain why this specific tool is needed or what prerequisites exist for updating runs within plan entries versus standalone runs.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_sectionC

Update an existing section

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesSection ID
nameNoSection name
descriptionNoSection description

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description fails to disclose behavioral traits such as whether updates are partial or full replacements, permissions needed, or side effects. Only the action 'update' is mentioned, which is insufficient for safe invocation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise at one sentence, but it is not wasteful. However, it could be improved by including key details without sacrificing brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has no output schema and no annotations, the description should explain what happens upon success or failure, or confirm return behavior. It does not, leaving the agent underinformed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for all three parameters. The description adds no additional meaning beyond what the schema already provides, earning a baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool updates an existing section, which is a specific verb+resource combination. However, it does not differentiate from sibling update tools like update_case or update_run, as all share similar generic phrasing.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus adding a section, updating other entities, or prerequisites like the section must exist. The description lacks any contextual cues for selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_suiteB

Update an existing test suite

ParametersJSON Schema
NameRequiredDescriptionDefault
suite_idYesSuite ID
nameNoSuite name
descriptionNoSuite description

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Since no annotations are provided, the description carries full burden. It only says 'Update,' implying mutation, but offers no details on side effects, authorization needs, error handling, or response behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no wasted words. However, it could include more useful information without being significantly longer.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no output schema and no annotations, the description omits critical context such as return values, error conditions, or whether the operation is idempotent. It is insufficient for an AI agent to fully understand the tool's behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with basic parameter descriptions. The tool description adds no additional meaning beyond the schema, so baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action (update) and the resource (existing test suite), distinguishing it from sibling tools like add_suite (create) and get_suite (read). It is unambiguous and specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives, such as update_case or update_run, nor does it mention prerequisites or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 77 tool updatesv1.0.3
    • First observedadd_attachment_to_case
    • First observedadd_attachment_to_plan
    • First observedadd_attachment_to_plan_entry
    • First observedadd_attachment_to_result
    • First observedadd_attachment_to_run
    • First observedadd_case
    • First observedadd_milestone
    • First observedadd_plan
    • First observedadd_plan_entry
    • First observedadd_result
    • First observedadd_result_for_case
    • First observedadd_results
    • First observedadd_results_for_cases
    • First observedadd_run
    • First observedadd_run_to_plan_entry
    • First observedadd_section
    • First observedadd_suite
    • First observedclose_plan
    • First observedclose_run
    • First observedcopy_cases_to_section
    • First observeddelete_attachment
    • First observeddelete_case
    • First observeddelete_cases
    • First observeddelete_milestone
    • First observeddelete_plan
    • First observeddelete_plan_entry
    • First observeddelete_run
    • First observeddelete_run_from_plan_entry
    • First observeddelete_section
    • First observedget_attachment
    • First observedget_attachments_for_case
    • First observedget_attachments_for_plan
    • First observedget_attachments_for_plan_entry
    • First observedget_attachments_for_run
    • First observedget_attachments_for_test
    • First observedget_case
    • First observedget_case_fields
    • First observedget_case_statuses
    • First observedget_case_types
    • First observedget_cases
    • First observedget_configs
    • First observedget_current_user
    • First observedget_history_for_case
    • First observedget_milestone
    • First observedget_milestones
    • First observedget_plan
    • First observedget_plans
    • First observedget_priorities
    • First observedget_project
    • First observedget_projects
    • First observedget_result_fields
    • First observedget_results
    • First observedget_results_for_case
    • First observedget_results_for_run
    • First observedget_run
    • First observedget_runs
    • First observedget_section
    • First observedget_sections
    • First observedget_statuses
    • First observedget_suite
    • First observedget_suites
    • First observedget_templates
    • First observedget_test
    • First observedget_tests
    • First observedget_user
    • First observedget_user_by_email
    • First observedget_users
    • First observedmove_cases_to_section
    • First observedmove_section
    • First observedupdate_case
    • First observedupdate_milestone
    • First observedupdate_plan
    • First observedupdate_plan_entry
    • First observedupdate_run
    • First observedupdate_run_in_plan_entry
    • First observedupdate_section
    • First observedupdate_suite

TDQS

B3/5.0

Scored across 77 tools

Disambiguation4/5

Most tools have distinct purposes targeting specific resources and actions, but there is some overlap that could cause confusion. For example, 'add_result' and 'add_result_for_case' are similar, and multiple attachment tools for different resources might be misselected if the agent isn't careful about the target entity.

Naming Consistency5/5

Tool names follow a highly consistent verb_noun pattern throughout, with clear actions like add, get, delete, update, copy, move, and close applied to specific nouns. This predictability makes it easy for agents to understand and navigate the toolset.

Tool Count2/5

With 77 tools, the count is excessive for typical MCP server purposes, making it overwhelming and difficult to manage. This large number suggests over-fragmentation of operations that could have been consolidated, such as multiple attachment tools for different resources.

Completeness5/5

The toolset provides comprehensive CRUD and lifecycle coverage for TestRail's domain, including projects, suites, cases, runs, plans, results, attachments, and users. There are no obvious gaps, with tools for creation, retrieval, updates, deletions, and specialized operations like moving or copying cases.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    Enables AI assistants to interact with TestRail test management systems through comprehensive API integration. Supports retrieving and updating test cases, projects, suites, runs, and results, plus adding attachments and managing test data through natural language commands.
    18
    12 npm
    MIT
  • A
    license
    Not graded
    quality
    D
    maintenance
    Enables AI assistants to interact with Testmo test management platform for creating, reading, updating, and deleting test cases, managing folders, and organizing test runs through natural language.
    4
    MIT
  • A
    license
    A
    quality
    F
    maintenance
    Enables management of TestRail projects, test cases, runs, and results directly through MCP-supported clients. It provides a comprehensive set of tools to interact with the TestRail API for seamless test cycle management within AI environments.
    42
    743 npm
    44
    MIT
  • A
    license
    B
    quality
    D
    maintenance
    Enables AI assistants to interact with TestRail test management system, supporting full CRUD operations on projects, suites, sections, test cases, runs, results, plans, and milestones.
    35
    1,504 npm
    1
    MIT