Skip to main content
Glama
g-tiwari

@g-tiwari/mcp-testrail

by g-tiwari

@g-tiwari/mcp-testrail

MCP server for the TestRail API. Gives AI assistants full access to manage test cases, runs, results, plans, and all other TestRail resources.

107 tools covering every TestRail API endpoint, with toolset filtering for token efficiency.

Works with any TestRail setup — standard cloud/self-hosted instances (API key or password auth) and enterprise SSO deployments (SAML/OIDC cookie-based auth).

Quick Start

Add to your MCP client config (Claude Desktop, Claude Code, Kiro, Cursor, Windsurf — see IDE Configuration below):

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://yourcompany.testrail.com",
        "TESTRAIL_EMAIL": "your@email.com",
        "TESTRAIL_API_KEY": "your-api-key"
      }
    }
  }
}

That's it. Restart your MCP client and start asking it to manage TestRail.

Enterprise SSO? Replace the API key env vars with "TESTRAIL_COOKIE_PATH": "~/path/to/cookies.txt" — see Authentication for all options. You can also add optional env vars like TESTRAIL_PROJECT_IDS, TESTRAIL_TOOLSETS, and TESTRAIL_READ_ONLY_MODE to the same env block — see Environment Variables.

Related MCP server: TestRail MCP Server

Authentication

This is the standard auth method for TestRail Cloud and self-hosted instances.

  1. Go to My Settings in TestRail (top-right menu)

  2. Enable the API under API section

  3. Generate an API key

Set these environment variables:

TESTRAIL_URL=https://yourcompany.testrail.com
TESTRAIL_EMAIL=your@email.com
TESTRAIL_API_KEY=your-api-key

Option 2: Password

For instances where API keys are not available.

TESTRAIL_URL=https://yourcompany.testrail.com
TESTRAIL_EMAIL=your@email.com
TESTRAIL_PASSWORD=your-password

For enterprise TestRail instances behind SSO providers (Okta, Azure AD, etc.). The server reads a Netscape-format cookie file, automatically completes the SAML/OIDC login flow, and obtains a TestRail session.

TESTRAIL_URL=https://testrail.yourcompany.com
TESTRAIL_COOKIE_PATH=~/path/to/sso/cookies.txt

How it works: The server loads SSO cookies from the file, hits the TestRail SSO redirect endpoint, parses the SAML response form, and submits it back to TestRail to obtain a tr_session cookie. It also hot-reloads when the cookie file changes (e.g. after re-authenticating with your SSO provider).

Note: You may need to refresh your SSO cookies periodically (e.g. re-authenticate via your SSO provider's CLI).

For enterprise setups where the SSO proxy requires cookies and TestRail also validates API key auth.

TESTRAIL_URL=https://testrail.yourcompany.com
TESTRAIL_COOKIE_PATH=~/path/to/sso/cookies.txt
TESTRAIL_EMAIL=your@email.com
TESTRAIL_API_KEY=your-api-key

IDE / AI Assistant Configuration

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

API Key auth:

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://yourcompany.testrail.com",
        "TESTRAIL_EMAIL": "your@email.com",
        "TESTRAIL_API_KEY": "your-api-key"
      }
    }
  }
}

SSO Cookie auth (enterprise):

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://testrail.yourcompany.com",
        "TESTRAIL_COOKIE_PATH": "~/path/to/sso/cookies.txt"
      }
    }
  }
}

Claude Code (CLI)

Add to ~/.claude/settings.json or your project's .claude/settings.json:

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://yourcompany.testrail.com",
        "TESTRAIL_EMAIL": "your@email.com",
        "TESTRAIL_API_KEY": "your-api-key"
      }
    }
  }
}

Kiro

Add to .kiro/settings/mcp.json in your project root:

API Key auth:

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://yourcompany.testrail.com",
        "TESTRAIL_EMAIL": "your@email.com",
        "TESTRAIL_API_KEY": "your-api-key"
      }
    }
  }
}

SSO Cookie auth (enterprise):

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://testrail.yourcompany.com",
        "TESTRAIL_COOKIE_PATH": "~/path/to/sso/cookies.txt"
      }
    }
  }
}

Cursor

Add to .cursor/mcp.json in your project root:

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://yourcompany.testrail.com",
        "TESTRAIL_EMAIL": "your@email.com",
        "TESTRAIL_API_KEY": "your-api-key"
      }
    }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "testrail": {
      "command": "npx",
      "args": ["-y", "@g-tiwari/mcp-testrail"],
      "env": {
        "TESTRAIL_URL": "https://yourcompany.testrail.com",
        "TESTRAIL_EMAIL": "your@email.com",
        "TESTRAIL_API_KEY": "your-api-key"
      }
    }
  }
}

Local Development

For local testing (not published to npm), use the absolute path to the built file:

{
  "mcpServers": {
    "testrail": {
      "command": "node",
      "args": ["/absolute/path/to/testrail-mcp/build/src/index.js"],
      "env": {
        "TESTRAIL_URL": "https://yourcompany.testrail.com",
        "TESTRAIL_EMAIL": "your@email.com",
        "TESTRAIL_API_KEY": "your-api-key"
      }
    }
  }
}

Environment Variables

Variable

Required

Description

TESTRAIL_URL

Yes

TestRail instance URL (e.g. https://yourcompany.testrail.com)

TESTRAIL_EMAIL

For API key/password auth

User email address

TESTRAIL_API_KEY

Option 1

API key (from My Settings in TestRail)

TESTRAIL_PASSWORD

Option 2

Password (if API keys are unavailable)

TESTRAIL_COOKIE_PATH

Option 3

Path to Netscape-format cookie file

TESTRAIL_TOOLSETS

No

Comma-separated toolset names or "all" (default: core toolsets)

TESTRAIL_TOOLS

No

Additional individual tool names to enable

TESTRAIL_DENIED_TOOLS_REGEX

No

Regex pattern to exclude tools by name

TESTRAIL_PROJECT_IDS

No

Comma-separated project IDs to scope the server to (e.g. "5" or "1,5,12")

TESTRAIL_READ_ONLY_MODE

No

Set to "true" to expose only read operations

If both TESTRAIL_API_KEY and TESTRAIL_PASSWORD are set, API key takes precedence.

Project Scoping

Use TESTRAIL_PROJECT_IDS to restrict the server to specific projects. This is useful when your TestRail instance has many projects but you only work with a few.

Configuration

Description hints

Auto-inject project_id

Restrict

Unset

None

No

No

Single ID ("5")

[Project 5] appended to tool descriptions

Yes — fills missing project_id

Rejects other IDs

Multiple IDs ("1,5,12")

[Allowed projects: 1, 5, 12] appended

No — LLM must pick

Rejects other IDs

# Scope to a single project (auto-injects project_id)
TESTRAIL_PROJECT_IDS="5"

# Scope to multiple projects (LLM must specify which)
TESTRAIL_PROJECT_IDS="1,5,12"

Tools like get_projects (no project_id input) and add_project (creates new) are unaffected. Entity-scoped tools like get_case (which take a case_id, not project_id) are also unaffected.

Toolsets

Tools are organized into 18 toolsets. 9 are enabled by default (54 tools) to keep token usage low. Use TESTRAIL_TOOLSETS to customize.

Toolset

Default

Tools

Description

projects

Yes

5

Project CRUD

suites

Yes

5

Suite CRUD

sections

Yes

6

Section CRUD + move

cases

Yes

13

Case CRUD + bulk + fields/types

runs

Yes

6

Run CRUD + close

tests

Yes

2

Read tests in runs

results

Yes

8

Result CRUD + bulk + fields

users

Yes

5

User lookup + roles

metadata

Yes

4

Priorities, statuses, templates

plans

No

12

Test plan management

milestones

No

5

Milestone CRUD

groups

No

5

Group management

attachments

No

8

File attachments

configurations

No

7

Test configurations

reports

No

2

Report execution

shared_steps

No

5

Shared steps

variables

No

4

Variables

datasets

No

5

Datasets

Toolset Examples

# Only case and result management (21 tools)
TESTRAIL_TOOLSETS="cases,results"

# Everything (107 tools)
TESTRAIL_TOOLSETS="all"

# Default toolsets + specific additional tools
TESTRAIL_TOOLS="get_plan,get_plans,get_milestones"

# Block all destructive operations
TESTRAIL_DENIED_TOOLS_REGEX="^(delete_|close_)"

# Read-only mode (~40 tools)
TESTRAIL_READ_ONLY_MODE="true"

Token Efficiency

Every MCP tool definition is injected into the LLM context on every conversation turn. More tools = more tokens burned before the model even reads your message. This MCP is designed to keep that cost low.

Default config uses ~2,000 tokens for tool definitions — roughly half of what exposing all 107 tools would cost (~4,000 tokens). Over a 10-turn conversation, that's ~20,000 tokens saved.

Configuration

Tools

~Tokens per turn

Default (9 core toolsets)

54

~2,000

Read-only mode

46

~1,700

Custom (e.g. cases,results)

21

~800

All toolsets

107

~4,000

How

  • Toolset filtering — only 54 of 107 tools exposed by default; non-essential toolsets (plans, milestones, groups, etc.) are opt-in

  • Compact descriptions — average 24 characters per tool (5-10 words), not full sentences

  • Bulk operationsadd_results, update_cases, delete_cases accept arrays, so one call replaces dozens of round-trips (each avoided round-trip saves the full tool list re-injection)

  • Max pagination — defaults to 250 items per page (TestRail's maximum), fewer calls needed

  • Raw JSON responses — no metadata envelope or wrapper overhead; delete/close operations return just {"success": true}

  • Layered filteringTESTRAIL_TOOLSETSTESTRAIL_TOOLSTESTRAIL_READ_ONLY_MODETESTRAIL_DENIED_TOOLS_REGEX all compose, so you can tune to exactly what you need

Rate Limiting

Automatic retry with exponential backoff on HTTP 429 responses. Respects the Retry-After header. Max 3 retries.

Available Tools (107)

Projects (5)

  • get_project — Get a project by ID

  • get_projects — List all projects

  • add_project — Create a new project

  • update_project — Update an existing project

  • delete_project — Delete a project by ID

Suites (5)

  • get_suite — Get a test suite by ID

  • get_suites — List suites for a project

  • add_suite — Create a test suite

  • update_suite — Update a test suite

  • delete_suite — Delete a test suite

Sections (6)

  • get_section — Get a section by ID

  • get_sections — List sections in a project/suite

  • add_section — Create a section

  • move_section — Move a section to a new parent

  • update_section — Update a section

  • delete_section — Delete a section

Cases (13)

  • get_case — Get a test case by ID

  • get_cases — List test cases with filters

  • get_history_for_case — Get change history for a case

  • add_case — Create a test case in a section

  • update_case — Update a test case

  • update_cases — Bulk update multiple test cases

  • delete_case — Delete a test case

  • delete_cases — Bulk delete multiple test cases

  • copy_cases_to_section — Copy cases to another section

  • move_cases_to_section — Move cases to another section

  • get_case_fields — List available case fields

  • add_case_field — Create a custom case field

  • get_case_types — List available case types

Runs (6)

  • get_run — Get a test run by ID

  • get_runs — List test runs for a project

  • add_run — Create a test run

  • update_run — Update a test run

  • close_run — Close a test run

  • delete_run — Delete a test run

Tests (2)

  • get_test — Get a test by ID

  • get_tests — List tests in a run

Results (8)

  • get_results — Get results for a test

  • get_results_for_case — Get results for a case in a run

  • get_results_for_run — Get all results for a run

  • add_result — Add a result for a test

  • add_result_for_case — Add a result for a case in a run

  • add_results — Bulk add results for a run

  • add_results_for_cases — Bulk add results by case for a run

  • get_result_fields — List available result fields

Plans (12)

  • get_plan — Get a test plan by ID

  • get_plans — List test plans for a project

  • add_plan — Create a test plan

  • add_plan_entry — Add a suite entry to a plan

  • add_run_to_plan_entry — Add a run to a plan entry

  • update_plan — Update a test plan

  • update_plan_entry — Update a plan entry

  • update_run_in_plan_entry — Update a run in a plan entry

  • close_plan — Close a test plan

  • delete_plan — Delete a test plan

  • delete_plan_entry — Delete a plan entry

  • delete_run_from_plan_entry — Delete a run from a plan entry

Milestones (5)

  • get_milestone — Get a milestone by ID

  • get_milestones — List milestones for a project

  • add_milestone — Create a milestone

  • update_milestone — Update a milestone

  • delete_milestone — Delete a milestone

Users & Roles (5)

  • get_user — Get a user by ID

  • get_current_user — Get the authenticated user

  • get_user_by_email — Find a user by email

  • get_users — List users

  • get_roles — List available user roles

Groups (5)

  • get_group — Get a group by ID

  • get_groups — List all groups

  • add_group — Create a user group

  • update_group — Update a user group

  • delete_group — Delete a user group

Attachments (8)

  • add_attachment_to_plan — Attach a file to a test plan

  • add_attachment_to_result — Attach a file to a result

  • add_attachment_to_run — Attach a file to a run

  • add_attachment_to_case — Attach a file to a case

  • get_attachments_for_case — List attachments for a case

  • get_attachments_for_run — List attachments for a run

  • get_attachment — Get attachment metadata by ID

  • delete_attachment — Delete an attachment

Configurations (7)

  • get_configs — List configurations for a project

  • add_config_group — Create a configuration group

  • add_config — Add a config to a group

  • update_config_group — Update a configuration group

  • update_config — Update a configuration

  • delete_config_group — Delete a configuration group

  • delete_config — Delete a configuration

Metadata (4)

  • get_priorities — List available priorities

  • get_statuses — List available test statuses

  • get_case_statuses — List available case statuses

  • get_templates — List templates for a project

Reports (2)

  • get_reports — List reports for a project

  • run_report — Execute a report template

Shared Steps (5)

  • get_shared_step — Get a shared step by ID

  • get_shared_steps — List shared steps for a project

  • add_shared_step — Create a shared step

  • update_shared_step — Update a shared step

  • delete_shared_step — Delete a shared step

Variables (4)

  • get_variables — List variables for a project

  • add_variable — Create a variable

  • update_variable — Update a variable

  • delete_variable — Delete a variable

Datasets (5)

  • get_dataset — Get a dataset by ID

  • get_datasets — List datasets for a project

  • add_dataset — Create a dataset

  • update_dataset — Update a dataset

  • delete_dataset — Delete a dataset

Troubleshooting

  • API Key auth: Verify your API key is correct and the API is enabled in TestRail (Administration > Site Settings > API).

  • SSO Cookie auth: Your SSO cookies may have expired. Refresh them (re-authenticate with your SSO provider) and try again. The server auto-detects cookie file changes.

"TESTRAIL_URL environment variable is required"

Ensure TESTRAIL_URL is set in your MCP config's env block. The URL should not have a trailing slash.

SSO login flow fails

  1. Ensure your SSO cookies are fresh and valid

  2. Test by opening your TestRail URL in a browser — if you can access it, the cookies should work

  3. Check that TESTRAIL_COOKIE_PATH points to the correct Netscape-format cookie file

  4. The server logs to stderr — check your MCP client's logs for detailed SSO flow output

Tools not appearing

Check TESTRAIL_TOOLSETS — by default only 9 core toolsets (54 tools) are enabled. Use TESTRAIL_TOOLSETS="all" for all 107 tools.

License

MIT

Available Tools

54 tools
add_caseB

Create a test case in a section

ParametersJSON Schema
NameRequiredDescriptionDefault
refsNoComma-separated references
titleYesCase title
type_idNoCase type ID
estimateNoEstimated time (e.g. '30s', '1m 45s')
section_idYesSection ID
priority_idNoPriority ID
template_idNoTemplate ID
milestone_idNoMilestone ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. It only states that a test case is created, but does not mention required existing resources, validation behavior, permissions, idempotency, or what happens on duplicate/malformed input.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, short sentence with no filler or redundancy. The action and resource are front-loaded, making it easy to scan and understand.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no annotations and no output schema, the description does not explain what the tool returns, whether a section must be valid, or how the optional parameters affect the created case. The schema covers parameters well, but the overall behavioral and return context is thin.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so each parameter already has a description. The tool description adds nothing beyond the general 'in a section' context, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create'), the resource ('a test case'), and the destination ('in a section'). This distinguishes it from related tools like update_case, delete_case, and add_result, making the tool's purpose immediately clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives, nor are prerequisites mentioned. The phrase 'in a section' implies a section must already exist, but this is not explicitly stated, and no exclusions or context are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_case_fieldC

Create a custom case field

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesField system name
typeYesField type (String, Integer, Text, URL, Checkbox, Dropdown, User, Date, Milestone, Steps, Multiselect)
labelYesField display label
configsNoField configuration per context
descriptionNoField description
include_allNoInclude in all projects
template_idsNoTemplate IDs to include field in

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only conveys that a custom field is created, but does not mention side effects, visibility across projects, whether existing fields are affected, or what happens on creation. The mutation is implied by 'Create' but no deeper behavior is disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single clear sentence with no filler or repetition. It is appropriately concise, though it sacrifices useful context that could have been added without bloating the description.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a create operation with 7 parameters, no annotations, and no output schema, the description is too thin. It does not explain required inputs, how configs and include_all interact, or what the successful response will be, leaving the agent to rely entirely on the schema and inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters. The description adds little meaning beyond the schema, but the baseline of 3 applies because the schema carries the parameter documentation burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the action ('Create') and the resource ('custom case field'), which is specific enough to distinguish it from siblings like add_case and get_case_fields. It does not further explain that this creates the field definition rather than setting a field value, but the core purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives, such as get_case_fields for listing existing fields or add_case for adding a test case. The description provides no context for selection or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_projectA

Create a new project

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesProject name
suite_modeNoSuite mode (1=single, 2=single+baselines, 3=multiple)
announcementNoProject description/announcement
show_announcementNoShow announcement on overview page

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full disclosure burden. It only indicates that this is a mutating operation, but it does not mention side effects, permissions, return values, uniqueness constraints, or what happens on success or failure.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise and front-loaded: 'Create a new project' contains no filler and every word adds meaning. It is appropriately sized for a simple create operation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple creation tool with complete schema coverage and no output schema, the description is minimally adequate. However, it lacks behavioral context, usage guidance, and any indication of what the tool returns, so an agent must rely on inference for several operational aspects.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides descriptions for all four parameters, including the required 'name' and optional 'suite_mode', 'announcement', and 'show_announcement'. The description adds no parameter-level detail beyond what the schema provides, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Create') and a specific resource ('a new project'), clearly distinguishing this from sibling tools like update_project, delete_project, and get_project. The purpose is unambiguous even without additional context.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The use case is only implied by the verb 'Create' — an agent can infer it is for creating a new project, but there is no explicit guidance about when to choose this tool over alternatives like update_project. No prerequisites, exclusions, or alternative routing are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_resultC

Add a result for a test

ParametersJSON Schema
NameRequiredDescriptionDefault
commentNoResult comment/notes
defectsNoComma-separated defect IDs
elapsedNoTime spent (e.g. '30s', '1m 45s')
test_idYesTest ID
versionNoVersion or build tested
status_idYesStatus ID (1=Passed,2=Blocked,3=Untested,4=Retest,5=Failed)
assignedto_idNoReassign to user ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only restates the action and gives no information about side effects, whether results are appended or replaced, permission requirements, or what the response contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no wasted words and is easy to parse. It is appropriately concise, though it achieves this by omitting useful contextual details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutating tool with no annotations, no output schema, and close sibling tools, the description is too thin. An agent lacks information about what a successful call returns, how this differs from related endpoints, and any behavioral implications of adding a result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all seven parameters are already explained in the schema, including the status_id value mapping. The description adds no extra parameter context beyond the schema, which fits the baseline score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Add') and a clear resource ('a result for a test'), so an agent can grasp the core operation. However, it does not distinguish this tool from closely named siblings like add_result_for_case or add_results, leaving some selection ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as add_result_for_case or add_results_for_cases. It also does not mention any prerequisites, constraints, or context in which this endpoint is the appropriate choice.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_result_for_caseC

Add a result for a case in a run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesTest run ID
case_idYesTest case ID
commentNoResult comment/notes
defectsNoComma-separated defect IDs
elapsedNoTime spent (e.g. '30s', '1m 45s')
versionNoVersion or build tested
status_idYesStatus ID (1=Passed,2=Blocked,3=Untested,4=Retest,5=Failed)
assignedto_idNoReassign to user ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. It merely says 'Add', which implies mutation, but it does not disclose side effects, required preconditions (e.g., run/case must exist), idempotency, or whether a result can be overwritten. This is a significant gap for a write operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. It is appropriately concise, though it does not add much beyond the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a mutating tool with 8 parameters, no output schema, and no annotations, yet the description offers only the bare action. An agent would not know what happens after calling it, whether the operation is reversible, or what constraints exist beyond the schema. The description is not complete enough for safe autonomous invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all 8 parameters are already documented in the schema. The description adds no additional parameter-level meaning. Per the baseline, this is acceptable at a score of 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Add a result') and a specific target ('for a case in a run'). It is clear on what the tool does, but it does not explicitly differentiate itself from closely related siblings like add_result, add_results, or add_results_for_cases, so it stops short of full distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to choose this tool over alternatives. The singular 'a case' implies single-case use, but no explicit conditions, exclusions, or comparisons to add_result, add_results, or add_results_for_cases are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_resultsB

Bulk add results for a run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesTest run ID
resultsYesArray of results

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations supplied, the description is the only behavioral cue; it discloses only that this is an append-style bulk create operation. It does not say whether existing results are replaced, how partial failures are handled, whether the run must be in a certain status, or what the response contains.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short, front-loaded sentence with no filler or redundancy. Every word contributes to identifying the operation and its scope.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The rich input schema covers parameter semantics, so the minimal description is not crippling for invocation. Still, without annotations or any behavioral notes (mutating effect, idempotency, return value), the definition leaves meaningful gaps for a write operation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents all parameters with 100% coverage, including status ID meanings and elapsed time format. The description adds no parameter-specific detail beyond the word 'bulk,' so the schema earns the baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the action (bulk add), the object (results), and the target (a run), and 'bulk' separates it from the single-result sibling add_result. It could further name the distinction from add_results_for_cases, but the core purpose is unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

'Bulk' implies this is for adding multiple results at once rather than using add_result, and 'for a run' ties it to run-scoped result entry. However, it does not explicitly state when to prefer this over add_results_for_cases or mention any prerequisites such as the run existing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_results_for_casesC

Bulk add results by case for a run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesTest run ID
resultsYesArray of results by case

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the burden of disclosing behavior. It only says 'Bulk add', a mutation, but does not mention whether results are appended or overwritten, what permissions are required, whether partial failures are possible, or what the response contains. This is a significant gap for a write operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, efficient sentence with no redundant wording and the core information front-loaded. It could contain more detail, but it is not padded or unstructured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The schema is rich and describes every parameter, but the tool has no output schema and no annotations. The description does not explain behavior, return value, error cases, or any restrictions. It is adequate for a straightforward bulk write with a self-explanatory schema, but noticeable gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already well documented. The description adds little beyond confirming the operation is bulk and keyed by case, which weakly connects to the run_id and results array. This meets the baseline for high schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear action ('Bulk add'), a specific resource ('results'), and the scope ('by case for a run'). It distinguishes itself from singular add_result_for_case by emphasizing 'bulk' and 'by case', though it does not explicitly differentiate from add_results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus siblings like add_result, add_result_for_case, or add_results. The context is only implied by 'Bulk add... by case', and there are no exclusions or alternative recommendations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_runC

Create a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesRun name
refsNoComma-separated references
case_idsNoCase IDs (when include_all=false)
suite_idNoSuite ID (for multi-suite projects)
project_idYesProject ID
descriptionNoRun description
include_allNoInclude all test cases
milestone_idNoMilestone ID
assignedto_idNoUser ID to assign

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure, and it only says 'Create a test run.' It does not state permissions required, what the call returns (e.g., a run object or ID), whether include_all defaults to true, or how the create operation interacts with refs/case_ids. The mutation implication is present but nothing beyond it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is maximally short and verb-fronted with no filler words, but for a tool with 9 parameters it is borderline under-specified rather than efficiently concise. It is not as sparse as a pure tautology, but it does not add informational value beyond the bare purpose statement.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with 9 parameters, no output schema, and no annotations, the description is inadequate. The agent is left without knowledge of return values, default behaviors (e.g., include_all), conditional parameter interactions, or error conditions. The schema documents the parameters well, but the description adds no operational context around the create action.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so every parameter (name, refs, case_ids, suite_id, project_id, description, include_all, milestone_id, assignedto_id) is already documented in the schema. The description adds no parameter-level meaning, so the baseline 3 applies; it neither helps nor hurts.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Create a test run' gives a specific verb ('Create') and a clear resource ('test run'), and the verb distinguishes it from sibling run operations like get_run, update_run, close_run, and delete_run. It is clear but minimal — 'test run' is domain jargon that is not elaborated, so it stops short of a top score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives. There is no mention of when creating a run is appropriate, no exclusions, and no reference to sibling tools like update_run or close_run. The verb 'create' only weakly implies the use case.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_sectionC

Create a section

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesSection name
suite_idNoSuite ID (for multi-suite projects)
parent_idNoParent section ID
project_idYesProject ID
descriptionNoSection description

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'Create a section,' which is essentially the same as the tool name and gives no information about side effects, required project/suite context, permissions, or the shape of the result. This is minimally transparent for a mutation operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short and front-loaded, with no wasted words. However, it is under-specified rather than efficiently comprehensive: it lacks any contextual or behavioral detail that would make the brevity genuinely useful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given that this is a creation tool with no annotations and no output schema, the description is too sparse. It does not explain how sections relate to projects or suites, whether parent_id is typical, or what the agent should expect after calling the tool. The schema helps with parameters, but the description leaves important operational context missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all five parameters clearly. The description adds no parameter-level meaning beyond what the schema provides, but it also does not need to compensate because the schema is complete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb and resource: 'Create a section.' It communicates the core operation immediately and is distinct from sibling tools like add_suite or add_project due to the resource term. However, it does not clarify whether this is a top-level section or a nested/hierarchical section, which would help further disambiguate from tools like move_section.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives such as update_section, delete_section, or move_section. The description implies it is for initial creation only, but it does not state prerequisites, when not to use it, or how it relates to parent sections or suites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_suiteB

Create a test suite

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYesSuite name
project_idYesProject ID
descriptionNoSuite description

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden, but it only says a test suite is created. It does not disclose side effects, permission requirements, whether project_id must reference an existing project, duplicate-suite behavior, or what response or error to expect.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with zero filler. It conveys the core operation immediately and lets the schema carry the field-level details, which is appropriate for this simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple create operation with fully documented scalar parameters, the description and schema provide the minimum needed to invoke the tool. However, with no annotations and no output schema, it omits practical context such as the need for an existing project and expected success or error behavior, so it is not fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions cover 100% of parameters (name, project_id, description) with basic field-level meaning, so the score sits at the baseline. The description itself adds no parameter semantics, but the schema already documents each field adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a precise action ('Create') and a precise resource ('a test suite'), clearly distinguishing this tool from sibling add_* tools like add_project, add_section, and add_case, as well as from suite read/update/delete tools. Even without the tool name, an agent can tell exactly what operation this performs.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to choose this tool over alternatives such as update_suite, delete_suite, or get_suites, nor any mention of prerequisites like an existing project_id. The only usage indication is the verb 'Create', which the agent must interpret without additional context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

close_runC

Close a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesTest run ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations present, the description carries full behavioral disclosure burden but only repeats the action without explaining side effects. It does not state whether closing is irreversible, whether it impacts result submission, or what state the run transitions into.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no unnecessary words. It is concise but borders on merely restating the tool name, and it lacks the extra detail that would make the conciseness genuinely informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Although the tool is simple with one required parameter and no output schema, the description lacks important context about the consequences of closing a run. Since there are no annotations to convey safety or side effects, the description alone is too thin for confident use beyond the most obvious invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the sole parameter run_id is already documented as 'Test run ID'. The description adds no additional semantic meaning or constraints beyond what the schema provides, which matches the baseline for full coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb ('close') and resource ('a test run'), which conveys the essential operation. It does not, however, explicitly differentiate itself from siblings like update_run or delete_run, leaving the agent to infer the distinction from the wording alone.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use close_run versus update_run or delete_run. There is no mention of prerequisites, such as the run needing to be active, or whether closing has finality implications that should make the agent prefer another tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

copy_cases_to_sectionB

Copy cases to another section

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idsYesArray of case IDs to copy
section_idYesSection ID

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only restates the basic copy action and does not explain side effects such as whether new case IDs are created, whether attachments or steps are copied, or what the response contains. The word 'copy' implies originals remain, but this is minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler or redundant wording. Every word contributes to identifying the operation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple two-parameter operation, the description covers the basic action and destination. However, it lacks explicit guidance on how it differs from the closely related 'move_cases_to_section' tool and does not disclose behavioral details like duplication semantics or response behavior. These gaps keep it at an adequate but incomplete level.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with both 'case_ids' and 'section_id' already documented in the input schema. The description adds no additional parameter meaning beyond the schema, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Copy'), names the resource ('cases'), and indicates the destination ('another section'). It inherently distinguishes from the sibling 'move_cases_to_section' by communicating a non-destructive copy operation rather than a move.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus its sibling 'move_cases_to_section'. The description does not state that this tool should be used when the original cases should remain in their source section, nor does it mention any alternatives or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_caseB

Delete a test case

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesTest case ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description is the only source of behavioral context, but it only says deletion occurs. It does not disclose irreversibility, cascading effects on tests or results, or permission requirements for a destructive operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The single sentence is minimal and front-loaded, with no filler. It is appropriately sized for a simple single-parameter deletion tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The one-line description is enough to understand the basic operation, but with no annotations and no output schema, a destructive tool like this needs at least a warning about permanence or side effects. The lack of any caveat leaves an agent without enough context for safe selection.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents case_id. The description adds no parameter-level information; the baseline 3 applies because the schema carries the burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action and resource: 'Delete a test case.' It clearly distinguishes the singular delete_case from the sibling delete_cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool instead of alternatives. There is no mention of delete_cases for bulk deletion or any exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_casesB

Bulk delete multiple test cases

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idsYesArray of case IDs to delete
suite_idNoSuite ID (required for multi-suite projects)
project_idYesProject ID

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries full responsibility for behavioral disclosure. It conveys that deletion is bulk and destructive, but it does not state that deletion is permanent, whether it cascades to related records like test results, whether permissions are required, or how the optional suite_id affects behavior. For a destructive operation, this is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short and front-loaded, stating the action and object immediately. 'Bulk' and 'multiple' are slightly redundant, so it is not perfectly economical, but there is no filler or unnecessary detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive bulk operation with no output schema and no annotations, the description is too thin. It does not explain return behavior, confirmations, error conditions, the meaning of suite_id in multi-suite projects, or the permanence of deletion. An agent invoking this tool has little context to anticipate side effects or handle failures.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents project_id, case_ids, and suite_id. The description adds no new parameter-level meaning beyond signaling bulk deletion; the baseline of 3 is appropriate because structured documentation handles the parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the verb ('delete'), the resource ('test cases'), and the scope ('bulk/multiple'), so an agent can distinguish it from the singular sibling delete_case and from update-type case tools. The resource and action are unambiguous even without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when deleting multiple test cases at once, but it gives no explicit guidance on when to prefer this tool over delete_case or when the optional suite_id is needed. It does not mention any alternatives or exclusions, leaving the agent to infer the usage boundary from sibling names.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_projectB

Delete a project by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It states that the operation deletes a project, but it does not disclose permanence, cascading deletion of associated objects, permission requirements, or return behavior. For a destructive action, this is a significant transparency gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no filler, places the action first, and immediately identifies both the resource and the required parameter. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter tool with full schema coverage, the description is minimally adequate for invoking the tool. However, it lacks context around destructive consequences, error conditions, and behavior on associated records, which matters for a delete operation with no annotations or output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%: project_id is fully documented as a required number. The description only echoes 'by ID' and adds no extra meaning beyond the schema, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Delete'), a clear resource ('project'), and the selection method ('by ID'). This distinguishes it from siblings like update_project, get_project, and delete_section without requiring the agent to open the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives, no mention of prerequisites, and no exclusions. The agent must infer usage entirely from the tool name and generic description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_runB

Delete a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesTest run ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations are absent, so the description must carry the burden of behavioral disclosure. While 'Delete' clearly signals a destructive action, it does not mention irreversibility, potential cascading effects on associated tests or results, permission requirements, or what happens after deletion.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single clear sentence with no wasted words. The verb and resource are front-loaded, making the tool's purpose immediately obvious.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive operation with no annotations and no output schema, the one-sentence description is insufficient. It omits important context such as side effects, return behavior, and consequences, although the single required parameter is fully documented by the schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, and the single parameter run_id is already described in the schema as 'Test run ID.' The tool description adds no additional meaning beyond what the schema provides, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Delete') and a clear resource ('a test run'), which unambiguously identifies the operation. This also distinguishes it from sibling delete tools like delete_case, delete_suite, and delete_project by naming the distinct resource type.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives such as close_run, update_run, or other run-related tools. The description simply states the action without any context, exclusions, or decision criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_sectionC

Delete a section

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesSection ID

TDQS

C2.6/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but 'Delete a section' merely restates the tool name. It does not mention that deletion is permanent/irreversible, whether child sections or contents are removed, or what response is returned. For a destructive operation this is a serious gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and readable, with no wasted words. However, it is essentially a tautology of the tool name and provides no extra value, so it is concise but not genuinely informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive mutation tool with no annotations and no output schema, the description is incomplete. It omits the irreversibility and scope of deletion, which are key for an agent to safely invoke it. The simple one-parameter schema does not compensate for that missing behavioral context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the single section_id parameter fully with a 'Section ID' description, so the 100% schema coverage sets the baseline at 3. The description does not add meaning beyond the schema, but none is needed for a single obvious parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource ('Delete a section') and is distinct from sibling delete tools like delete_suite and delete_case. However, it adds no scope or context beyond the tool name, so it stops short of fully distinguishing behavior.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to choose this tool over alternatives or any prerequisites or consequences of deletion. The agent is left to infer from the name and sibling set that this is for deleting a section rather than moving or updating it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_suiteC

Delete a test suite

ParametersJSON Schema
NameRequiredDescriptionDefault
suite_idYesSuite ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description bears the full burden of behavioral disclosure. 'Delete a test suite' only conveys that the operation is destructive; it does not disclose whether deletion is permanent, whether it cascades to contained tests, or whether special permissions are required.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise phrase with no filler or redundancy. It is appropriately brief for a simple tool, though it leans closer to under-specification than to richly informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a deletion tool with no annotations and no output schema, the description is minimal. It provides enough to identify the operation and required parameter, but an agent is left without information about side effects, reversibility, or relationships to related entities like test cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% since the only parameter, suite_id, is described as 'Suite ID'. The tool description adds no further meaning, so it does not exceed the baseline established by the well-covered schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Delete') and identifies the target resource ('test suite'), which is clear enough to distinguish it from sibling tools like delete_section, delete_project, and delete_case. It is concise and unambiguous, though it does not add contextual detail beyond restating the operation implied by the tool name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives such as get_suite, update_suite, or delete_section. The description simply states the action without providing context, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_caseB

Get a test case by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idYesTest case ID

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of disclosing behavior. 'Get' implies a read-only, non-destructive operation, which is useful, but the description does not disclose return behavior, error handling, or permissions. This is acceptable for a simple fetch but not richly transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no wasted words. It communicates the essential operation immediately.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple, with one parameter and no output schema, so the description is minimally viable. However, it does not mention what is returned, any limitations, or how this relates to similar tools, leaving some context missing for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the case_id parameter is already documented. The description's 'by ID' aligns with the parameter but adds no additional semantic meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb ('Get'), the resource ('a test case'), and the lookup key ('by ID'). It is distinct from get_cases, which implies a list operation, though it does not explicitly contrast with get_test or other case-related siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool versus alternatives. The intended usage is only implied by the phrase 'by ID,' and no mention is made of when to prefer get_cases, get_test, or listing endpoints.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_case_fieldsB

List available case fields

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the full burden. 'List' implies a read-only operation, but the description does not disclose what a 'case field' is, whether both built-in and custom fields are returned, or any response format details. Minimal but not misleading.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no filler. Every word contributes to the meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter tool with no output schema, the description is nearly sufficient, but it leaves unclear what the returned 'fields' consist of (names only, structured objects, metadata). An agent invoking the tool could not anticipate the response shape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and schema coverage is 100% (empty object), so the baseline is 4. The description adds no parameter-specific meaning, but none is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('List') and resource ('case fields'), which clearly identifies the operation. The phrase 'available' is slightly vague, but it distinguishes the tool from siblings like get_case, get_case_types, and get_result_fields.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides no guidance on when to use this tool versus alternatives such as get_result_fields or add_case_field. With a long list of sibling tools, the description does not position this tool or mention exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_casesB

List test cases with filters

ParametersJSON Schema
NameRequiredDescriptionDefault
refsNoComma-separated reference IDs
limitNoMax items (max 250, default 250)
filterNoFull text search filter
offsetNoOffset for pagination (default 0)
type_idNoComma-separated case type IDs
suite_idNoSuite ID (required for multi-suite projects)
status_idNoComma-separated status IDs (1=Passed,2=Blocked,3=Untested,4=Retest,5=Failed)
created_byNoComma-separated creator user IDs
project_idYesProject ID
section_idNoSection ID filter
updated_byNoComma-separated updater user IDs
priority_idNoComma-separated priority IDs
template_idNoComma-separated template IDs
milestone_idNoComma-separated milestone IDs
created_afterNoCreated after timestamp
updated_afterNoUpdated after timestamp
created_beforeNoCreated before timestamp
updated_beforeNoUpdated before timestamp

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It tells the agent this is a list/read operation with filters, which implies non-destructive behavior, but it does not disclose pagination defaults, response shape, or the fact that no output schema exists. The description is safe but thin on behavioral specifics such as limit default and maximum.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single short sentence with no filler, and the core action 'List test cases' is front-loaded. It is concise enough, though it omits useful context that could be added without bloating the text, such as 'use suite_id for multi-suite projects'.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 18 parameters, no output schema, no annotations, and no usage guidance, the description is minimal. It covers the basic purpose but does not explain pagination behavior, the multi-suite requirement, or how filtering parameters combine. The schema covers parameter semantics, but an agent would benefit from a sentence about pagination and response structure for a list operation of this complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the schema already documents all 18 parameters and their meanings. The description adds no additional parameter nuance beyond 'filters', and it does not mention relationships like suite_id being required for multi-suite projects. With full schema coverage, a baseline of 3 is appropriate, but the description misses a chance to clarify filter semantics like comma-separated values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'List test cases with filters' clearly identifies a list operation on the test-cases resource and signals that filtering is supported. It is brief but sufficient to distinguish it from get_case (singular) and from add_case/update_case/delete_case siblings. It could be stronger by naming the project context or noting pagination, but it is not tautological or vague.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this is the go-to tool for listing test cases, and the schema shows required project_id plus a suite_id required for multi-suite projects. However, it does not explicitly state when to prefer this over get_case, get_test, or get_results, nor does it mention the suite_id prerequisite for multi-suite projects. Usage context is implied rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_case_statusesB

List available case statuses

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of disclosing behavior. 'List' implies a read-only retrieval with no side effects, which is helpful, but the description does not explain whether statuses are system-defined or configurable, whether any ordering applies, or whether there are access considerations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four words and conveys the operation and resource with no filler. It is front-loaded and every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple parameterless list tool, the description is mostly adequate, but the lack of differentiation from the similarly named get_statuses sibling leaves a real selection ambiguity. An agent may not know whether get_case_statuses is the correct choice without additional context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there are no schema gaps to compensate for. The description adds useful semantic context by specifying that the resource is 'case statuses', even though no parameter-level detail is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific verb (List) and resource (case statuses), so an agent can understand what the tool does. However, it does not distinguish get_case_statuses from the sibling tool get_statuses, leaving some ambiguity about the exact scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus alternatives such as get_statuses, get_priorities, or get_case_types. An agent must infer the intended context purely from the tool name.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_case_typesB

List available case types

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral burden. 'List' clearly indicates a read-only operation and 'available' scopes the returned set, but nothing is disclosed about response shape, ordering, authentication, or other behavior beyond that basic implication.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single four-word sentence with the verb and resource front-loaded. Every word earns its place and there is no filler or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter list call, this is nearly adequate, but with no output schema and no annotations, the agent still has to guess what a case type object contains and which sibling to prefer when several metadata listers exist. A brief note that the result is usable for case creation would make it complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters and an empty input schema, so there is no parameter-level meaning for the description to add. This matches the baseline for no-parameter tools.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and a resource ('available case types'), making it clear this is a read-only enumeration. However, it does not explicitly contrast it with sibling metadata listers like get_case_statuses or get_templates, so differentiation relies solely on the resource name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives. An agent is not told how case types relate to case creation or whether another sibling tool would be more appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_current_userB

Get the authenticated user

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Get' implies a read-only operation, but the description does not address what happens when no user is authenticated, what the response contains, or any error conditions. With no output schema present, the return shape is entirely unspecified.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

At five words, the description is maximally concise and front-loads the core purpose immediately. Every word earns its place, and for a zero-parameter tool this brevity is appropriate rather than wasteful. It could add a clarifying detail, but nothing here is padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (zero parameters, no nested objects), the description conveys the essential operation adequately. However, with no output schema and no annotations, the response format and authentication-failure behavior are left unspecified — an agent cannot know what fields the user object contains or how errors surface. This is a meaningful gap, though minor for a widely understood pattern.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has zero properties and 100% schema description coverage, so there are no parameters for the description to clarify. Per the zero-parameter baseline, the description is not required to add parameter-level semantics, and the phrase 'the authenticated user' correctly implies the call takes no inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get') and resource ('the authenticated user'), clearly stating the tool returns the profile of the currently authenticated user. The word 'authenticated' implicitly distinguishes it from sibling lookups like get_user, get_users, and get_user_by_email, though that differentiation is not made explicit. Clear purpose, but it relies on inference rather than naming the distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The single-sentence description offers no guidance on when to use this tool versus alternatives. With 56 siblings including get_user, get_users, and get_user_by_email, an agent receives no explicit signal about the deciding condition (session-based lookup vs. ID vs. email). No exclusions, prerequisites, or alternative references are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_history_for_caseA

Get change history for a case

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
case_idYesTest case ID

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. 'Get' makes clear this is a read operation, but the description does not disclose whether the history is ordered, paginated (though limit/offset are in the schema), or what types of changes are included. It is minimally adequate but lacks richer behavioral context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler or redundancy. Every word contributes to identifying the operation. This is an ideal level of conciseness for such a simple retrieval tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple three-parameter tool, the description plus a fully documented schema is minimally viable. However, there is no output schema and the description does not describe the shape or content of the returned history, nor does it provide guidance relative to sibling tools. An agent could invoke it correctly, but it might not fully anticipate what the response contains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters (case_id, limit, offset) are already documented with descriptions. The tool description adds no semantic information beyond the schema, which matches the baseline score of 3. The schema carries the full parameter burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Get'), a specific resource ('change history'), and the target object ('a case'). It is clearly distinct from sibling tools like get_case or get_results_for_case, which return current case data or test results rather than case change history. No ambiguity remains about which tool to use for this purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the use case: call this tool when you need the change history of a particular case. However, it does not explicitly mention when not to use it, nor does it point to alternatives such as get_case for current case details or get_results_for_case for test results. The usage context is understandable but left mostly to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_prioritiesB

List available priorities

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description bears the behavioral disclosure burden. 'List' implies a read-only operation, but there is no mention of ordering, pagination, permissions, error behavior, or the returned priority object structure, and no output schema exists to compensate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three words, front-loaded with the action and resource, and contains no filler or redundant detail. Every word contributes to meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a parameterless list operation, this is minimally usable: it tells the agent that calling the tool returns available priorities. However, without annotations or an output schema, it does not describe the priority object shape or the exact meaning of 'available', which could matter for downstream processing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so there is nothing for the description to add about parameter meaning. The schema fully covers the empty input, and the 0-parameter baseline of 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and a concrete resource ('priorities'), making the core purpose clear. It stops short of a 5 because it does not clarify scope—whether these are all priorities globally or project-specific—or what fields a priority object contains.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus similar sibling list tools such as get_statuses, get_case_statuses, or get_templates. No alternatives or exclusions are mentioned, leaving the agent to infer when 'priorities' is the right resource.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_projectA

Get a project by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The verb 'Get' signals a read-only retrieval with no destructive side effects, which is the core behavioral trait. However, no annotations are provided, so the description is the only safety signal; it omits details about return behavior, error cases, or authentication requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single five-word sentence with no filler words. It is appropriately sized for a simple single-parameter read operation, and the key qualifier 'by ID' is included without unnecessary explanation.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a minimal CRUD read with one required parameter and no output schema, the description provides the basic contract but not enough detail. It does not describe the response shape, error behavior, or how it differs from get_projects, leaving moderate gaps for an agent to discover.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% because project_id is documented as 'Project ID' in the input schema. The tool description adds no further parameter meaning beyond restating retrieval by ID, which meets the baseline of 3 for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get a project by ID' states a specific verb ('Get'), a clear resource ('project'), and a qualifier ('by ID') that distinguishes it from sibling get_projects. An agent can immediately identify this as a single-resource fetch rather than a list operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus get_projects, update_project, or delete_project. There are no stated alternatives, prerequisites, or exclusions, so usage context must be inferred entirely from the tool name and schema.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_projectsC

List all projects

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
is_completedNo1=completed, 0=active

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full disclosure burden. 'List all projects' implies an unbounded result, but the schema shows a default limit of 250, offset pagination, and an is_completed filter, none of which are mentioned. This is a meaningful behavioral gap for an agent deciding whether to call it.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely short and front-loaded, with no wasted words. However, it is terse to the point of omitting useful behavioral context, so it earns a 4 rather than a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and no annotations, the description should compensate by clarifying pagination, filtering, and return conventions. It does none of this, and the word 'all' is actively misleading given the default limit of 250.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and all three parameters already have clear descriptions (limit, offset, is_completed). The description adds no new parameter meaning, but the baseline of 3 is appropriate when the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb+resource pair ('List all projects') and implies a plural listing, which distinguishes it from get_project (singular) and CRUD siblings. It is specific enough that an agent can infer the basic operation without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like get_project, nor any mention of context such as filtering or pagination needs. The description simply states the action without exclusions or conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_result_fieldsA

List available result fields

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

'List' clearly signals a non-mutating enumeration and identifies what is returned. However, with no annotations and no output schema, the description does not clarify whether both system and custom fields are returned, what per-field structure to expect, or whether the set of available fields is project-dependent. It is minimally transparent but not richly so.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence with no filler; every word ('List', 'available', 'result fields') earns its place, and the main predicate is front-loaded. Additional wording would mostly repeat API/context information rather than improve clarity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter metadata listing tool, 'List available result fields' is largely sufficient for an agent to select and invoke the call. There is no output schema, so the description necessarily carries the burden of outlining the return concept, and 'result fields' does that. It could be more explicit about result-field structure, but the low complexity keeps the gaps minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema declares zero parameters and schema description coverage is trivially 100%, so there are no parameter semantics to document. The baseline of 4 for a parameterless tool applies; the description does not need to add input-level detail.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a clear action ('List') and resource ('available result fields'), so an agent can infer that this endpoint returns result-field metadata. It distinguishes from siblings like get_case_fields and get_results through the 'result fields' scope, though it does not explicitly call out that distinction. It is not a pure tautology because it adds the 'List available' scope.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage context is only implied: call this when the agent needs available result fields. It does not provide explicit when-to-use vs. when-not-to-use guidance, nor does it mention alternatives such as get_case_fields for case fields. The resource name itself carries most of the routing information.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_resultsC

Get results for a test

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
test_idYesTest ID
status_idNoComma-separated status IDs (1=Passed,2=Blocked,3=Untested,4=Retest,5=Failed)
created_afterNoCreated after timestamp
updated_afterNoUpdated after timestamp
created_beforeNoCreated before timestamp
updated_beforeNoUpdated before timestamp

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. It only says 'Get results for a test' and reveals nothing about pagination, response structure, default behavior, or that this is a read-only operation beyond the verb 'get.'

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is minimal and free of filler, being a single clear sentence. It is concise, but it is also somewhat under-specified given the number of parameters and lack of additional context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, no annotations, and eight parameters including filters and pagination, the description is far too minimal to fully orient an agent. It does not explain return format, filtering semantics, or how results are ordered.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all eight parameters are already documented in the input schema. The description adds no additional meaning or context about how the parameters relate to the tool's behavior.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action and resource: getting results for a test. It does not explicitly distinguish itself from sibling tools like get_results_for_run or get_results_for_case, though the phrase 'for a test' narrows the target resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as get_results_for_case or get_results_for_run. The description only states what the tool does, leaving the agent to infer appropriate usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_results_for_caseA

Get results for a case in a run

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
run_idYesTest run ID
case_idYesTest case ID
status_idNoComma-separated status IDs (1=Passed,2=Blocked,3=Untested,4=Retest,5=Failed)
created_afterNoCreated after timestamp
updated_afterNoUpdated after timestamp
created_beforeNoCreated before timestamp
updated_beforeNoUpdated before timestamp

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of signaling side effects; 'Get' does indicate a read-only retrieval and there is no contradiction. However, it does not explain pagination defaults, whether multiple result entries per case/run are possible, or how the result set is organized, so behavioral insight beyond the verb is thin.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler or redundant restatement of the schema. Every word contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The one-line description is minimally viable for a simple query tool, especially with a fully documented schema, but it omits any explanation of returned shape or pagination behavior, and there is no output schema to compensate. It also does not mention how the optional filters combine, leaving the agent to infer behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents all nine parameters, including defaults and allowed values. The description adds only the conceptual mapping of run_id and case_id to 'a case in a run' and provides no additional parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('Get'), a concrete resource ('results'), and scopes it to 'a case in a run', which disambiguates it from siblings like get_results_for_run and get_case. This is a clear, distinguishing purpose statement.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for a case in a run' implies this tool should be used when the caller has both run_id and case_id and wants case-level results, but it never explicitly states when to prefer get_results_for_run or get_results instead. Usage must be inferred from the schema and sibling names rather than directly stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_results_for_runB

Get all results for a run

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
run_idYesTest run ID
status_idNoComma-separated status IDs (1=Passed,2=Blocked,3=Untested,4=Retest,5=Failed)
created_afterNoCreated after timestamp
updated_afterNoUpdated after timestamp
created_beforeNoCreated before timestamp
updated_beforeNoUpdated before timestamp

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the behavioral burden. 'Get' implies a read operation, but the description does not disclose pagination behavior, the default limit of 250, or that 'all results' requires offset/limit traversal. This is a meaningful gap for an 8-parameter tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

One short sentence with no filler. The core purpose is front-loaded and every word contributes. The brevity is a virtue here, though other dimensions penalize the missing detail it omits.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters, no output schema, and no annotations, this description is too thin. It does not explain pagination, timestamp formats, status filtering, or what a returned result object contains. An agent would need external knowledge to use the tool correctly at scale.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters are already documented. The description adds no extra meaning beyond run scoping, but the baseline of 3 applies because the schema carries the parameter semantics adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb and resource: 'Get all results for a run'. It identifies the run scoping, but it does not differentiate this from sibling tools like get_results or get_results_for_case, so it stops short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'for a run' implies the intended context, so an agent can guess when to use it. However, there is no explicit guidance about when to prefer this tool over get_results or get_results_for_case, and no mention of exclusions or alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_rolesB

List available user roles

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. The verb 'List' conveys a read-only operation, but the description does not clarify auth requirements, ordering, whether the result depends on the current user, or any other runtime behavior. It is adequate for a trivial read tool but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is four words with no filler, fully front-loaded, and every word contributes to meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter list operation, the description is mostly sufficient. However, there is no output schema and no annotations, so 'available' is ambiguous—it could mean all roles in the system or roles visible to the caller—and the response shape is not described.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the empty input schema leaves nothing undocumented. The description's phrase 'user roles' adds a bit of domain context, and the zero-parameter baseline of 4 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb and resource: 'List' + 'available user roles'. It is distinguishable from sibling tools like get_users and get_priorities by naming the 'roles' resource, though it does not explicitly contrast itself with those alternatives.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus related tools such as get_priorities or get_statuses, and there is no mention of whether 'available' depends on the caller's permissions. The intended context is only implied by the tool name and terse description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_runA

Get a test run by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
run_idYesTest run ID

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must carry the behavioral burden. 'Get' conveys read-only retrieval, but the description does not disclose response shape, not-found behavior, authentication needs, or any other caveats. It is safe but thin.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with zero filler. Every word contributes to identifying the operation and the key lookup criterion.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the low complexity—one required numeric parameter and no enums or nested objects—the description is largely sufficient. There is no output schema, so a note about the return value would improve completeness, but the semantics of 'get a test run by ID' already imply the retrieved run object.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already provides 100% coverage for the single parameter with 'Test run ID'. The description's 'by ID' adds no meaningful semantic information beyond what the schema states, so baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get') and names the exact resource ('test run') with a lookup qualifier ('by ID'). This clearly distinguishes it from siblings like get_runs (plural) and get_test (a different entity).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by ID' establishes the clear use case: retrieve a single run when you have its run_id. It does not explicitly name alternatives or exclusions, but for a one-parameter lookup the context is clear enough.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_runsB

List test runs for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
suite_idNoSuite ID filter
project_idYesProject ID
refs_filterNoReference filter
is_completedNo1=completed, 0=active
milestone_idNoComma-separated milestone IDs
created_afterNoCreated after timestamp
updated_afterNoUpdated after timestamp
created_beforeNoCreated before timestamp
updated_beforeNoUpdated before timestamp

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only says 'List.' It does not mention pagination behavior, maximum result size, ordering, return format, or that this is a read-only operation beyond what the verb implies.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It clearly communicates the action, resource, and scope while remaining appropriately compact.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has 11 parameters, no output schema, and no annotations, yet the description only provides a high-level listing statement. It does not explain the return shape, pagination semantics, filter behavior, or what fields are included in a listed run, leaving important operational context unspecified.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already documents all 11 parameters with descriptions, so the baseline is 3. The description only adds the context that runs are scoped 'for a project,' which aligns with the required project_id parameter but provides no additional semantic value beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'List test runs for a project.' The plural 'test runs' clearly distinguishes this from the singular 'get_run' sibling, and the project scoping matches the required project_id parameter.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as get_run, add_run, update_run, or close_run. There is no explicit when-to-use, when-not-to-use, or mention of alternatives, so an agent must infer usage from the verb alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_sectionB

Get a section by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
section_idYesSection ID

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the behavioral transparency burden. 'Get a section by ID' truthfully indicates a read operation but adds no detail about return format, error behavior, or read-only guarantees. It is not misleading, but it provides only the minimal behavioral info inherent in the verb.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence with no redundant words. Every token contributes meaning: 'Get' states the action, 'section' the resource, and 'by ID' the lookup method. It is appropriately sized for a simple getter.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple: one required parameter, no output schema, no annotations. The description covers the core operation, but it leaves an agent to infer what a 'section' is and what the response will contain. For a basic get-by-ID, this is minimally sufficient, but it lacks any richer context about output or error cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, and the schema already fully documents the single 'section_id' parameter. The description's 'by ID' adds no new semantic detail beyond restating the parameter's purpose. Baseline 3 applies because the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description 'Get a section by ID' uses a specific verb (get), resource (section), and lookup key (ID), making it clear this is a single-resource read. It is not a tautology and is distinguishable from the plural sibling get_sections, though it does not explicitly name the distinction.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no guidance on when to use this tool versus alternatives such as get_sections, add_section, update_section, or delete_section. There is no explicit exclusions or conditions; the only implied usage is 'when you have a section ID', which is not stated as guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_sectionsC

List sections in a project/suite

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
suite_idNoSuite ID (required for multi-suite projects)
project_idYesProject ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description must carry the full behavioral burden. 'List' implies a read operation, but the description does not disclose pagination behavior, the conditional need for suite_id, or what a response contains. The meaningful behavioral nuance about multi-suite projects is left entirely to the input schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very compact and contains no filler words. However, the slash in 'project/suite' is slightly ambiguous and could have been replaced with more explicit phrasing without adding much bulk.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no annotations, the description is thin for a tool that has pagination parameters and a conditional suite_id requirement. An agent would not understand the multi-suite nuance or the expected response shape from this description alone. The schema compensates for parameter details but not behavioral completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents all parameters and their defaults. The description adds only the hierarchical context 'in a project/suite', which is marginal and not required given the schema's completeness. A baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and resource ('sections') with a scope ('in a project/suite'), making the primary function clear. It does not explicitly differentiate from the sibling get_section, but the plural form and scope make the distinction reasonably inferable.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no guidance on when to use this tool versus alternatives such as get_section, get_suite, or get_suites. The only contextual hint is 'in a project/suite', which is too vague to direct an agent on selecting this tool for multi-suite projects or single-suite projects.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_statusesB

List available test statuses

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It clearly indicates a read-only operation ('List'), but it adds no detail about ordering, permissions, pagination, or the precise meaning of 'test statuses'. This is minimally adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single four-word sentence with no wasted words. It is front-loaded with the verb and clearly identifies the operation and object, making it ideal for a simple tool.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description is sufficient for a trivial zero-parameter list operation, but the existence of the closely named sibling 'get_case_statuses' introduces ambiguity that is not resolved. Without an output schema, slightly more detail about what 'test statuses' includes would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters and the schema is empty, so there are no parameter semantics to describe. The baseline for a 0-parameter tool is 4, and the description does not need to compensate for any schema gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a clear verb ('List') and a specific resource ('available test statuses'), so an agent can understand what the tool does. However, it does not differentiate from the similarly named sibling tool 'get_case_statuses', which prevents a higher score.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided about when to use this tool versus alternatives like get_case_statuses. There are no mentions of context, prerequisites, or exclusions, leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_suiteA

Get a test suite by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
suite_idYesSuite ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. 'Get' communicates a read-only fetch and no destructive side effects, but it does not describe response shape, missing-ID behavior, or access requirements. This is adequate for a simple getter but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence with no filler. Every word earns its place, and the key scoping detail ('by ID') appears early.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter getter with no nested objects or enums, this description, together with the schema, supplies enough to select and invoke the tool. It could optionally mention what the response contains since no output schema exists, but the resource name makes that largely predictable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the sole parameter is already described as 'Suite ID'. The description's 'by ID' phrase mirrors the schema and adds no additional meaning, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Get') and resource ('test suite') and scopes the operation to a single entity by ID. It naturally contrasts with sibling get_suites (plural), which implies the list version, so an agent can tell them apart without opening schemas.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by ID' implies the tool is used when a specific suite_id is known, but it gives no explicit guidance on when to prefer get_suites or other getters. Usage context is inferred rather than stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_suitesC

List suites for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that the tool lists suites; it does not disclose pagination, return format, ordering, scope limits (e.g., whether closed/archived suites are included), or authentication requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Five words with zero waste and a front-loaded verb. It is appropriately terse for a single-parameter read tool, though it sacrifices a small amount of informational density for brevity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter tool with no output schema and no annotations, the description covers the basic purpose but omits behavioral context such as pagination, what fields are returned, and how this differs from get_suite. Adequate for the simplest case, but a capable agent would still need to guess at return semantics.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The phrase 'for a project' loosely reinforces that project_id scopes the query, but the description adds no details beyond the schema's own 'Project ID' documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('List'), a clear resource ('suites'), and a scoping qualifier ('for a project'). It distinguishes itself from the mutation siblings (add_suite, update_suite, delete_suite) and, via the plural form, from get_suite, though that differentiation is implicit rather than explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus get_suite (listing all suites vs. retrieving one suite) or any other sibling. No context, exclusions, or alternatives are mentioned, leaving the agent to infer usage from the name alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_templatesA

List templates for a project

ParametersJSON Schema
NameRequiredDescriptionDefault
project_idYesProject ID

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of indicating safety and side effects. 'List' clearly implies a read-only operation with no destructive behavior, which is adequate for this simple tool. However, it does not disclose details such as return format, ordering, or permissions, which would add further transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence that states the action and scope with no filler. Every word earns its place, and the core purpose is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter list tool, the description is largely sufficient. It does not describe the response shape, but the tool name and description make the expected return value (templates) obvious. The absence of an output schema raises only minor ambiguity, so the description is nearly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% for the single project_id parameter, so the schema already fully documents the parameter. The description only reinforces that templates are scoped to a project without adding new semantic detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List'), a clear resource ('templates'), and a scope ('for a project'). There are no sibling template tools, so it is easily distinguishable from the provided siblings without requiring schema inspection.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly conveys when to call the tool: when the agent needs templates belonging to a project. It does not explicitly discuss exclusions or alternatives, but no template-related sibling tool exists, so the usage context is clear without additional routing guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_testA

Get a test by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
test_idYesTest ID

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only repeats the operation implied by the name ('Get') and does not disclose outcomes for missing IDs, authentication requirements, or other behavioral characteristics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler words. It conveys the essential purpose efficiently.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a one-parameter read operation, this is minimally adequate. However, without annotations or an output schema, the description does not hint at the return format or edge-case behavior, leaving some ambiguity for an agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with test_id documented as a number and described as 'Test ID'. The description adds no new parameter semantics beyond the schema, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the specific operation (get) on a specific resource (test) and the selection criterion (by ID). It clearly differentiates from the sibling get_tests, which likely lists tests, and from other get_* tools by naming the resource.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by ID' implies the tool should be used when a specific test_id is known, and the sibling get_tests exists for listing. However, there is no explicit guidance about when to choose this over alternatives or any exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_testsC

List tests in a run

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
run_idYesTest run ID
status_idNoComma-separated status IDs (1=Passed,2=Blocked,3=Untested,4=Retest,5=Failed)

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. 'List tests in a run' indicates a read operation but does not mention pagination behavior, default limits, status filtering, response contents, ordering, or any other runtime behavior an agent would need to anticipate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no filler or redundant content. It front-loads the core operation and scope without wasting tokens.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has four parameters, no output schema, no annotations, and many closely related sibling tools, yet the description only gives a minimal phrase. It does not explain what a returned test looks like, how status filtering interacts with the output, or how this differs from getting results or cases for a run, leaving an agent under-informed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% parameter coverage, so the baseline is 3 even though the description adds no parameter details. The description's 'in a run' reinforces run_id, but limit, offset, and status_id are already fully documented in the schema, so the description adds little beyond it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('List') and resource ('tests') with a clear scoping clause ('in a run'), so an agent understands the basic operation. It is distinct from singular get_test and from run-centric tools like get_run/get_runs, though it does not explicitly contrast with get_cases or get_results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to choose get_tests over related tools such as get_cases, get_results, or get_results_for_run. The phrase 'in a run' implies some context, but there are no explicit conditions, exclusions, or alternatives mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_userA

Get a user by ID

ParametersJSON Schema
NameRequiredDescriptionDefault
user_idYesUser ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure. 'Get' clearly signals a read-only operation, but the description does not mention return shape, error behavior, or other side effects. For a simple get-by-ID this is adequate but not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four words with zero redundancy, stating exactly the action and resource. All essential information is front-loaded and nothing extraneous is included.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter read tool, the definition is mostly sufficient. The only minor gap is that no output schema or return-value description is provided, so an agent must infer what a successful response contains.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already documents user_id as a required number with 100% coverage. The description adds little beyond restating the lookup key, so the schema does the heavy lifting and the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The verb 'Get' plus the resource 'user' and the access method 'by ID' clearly define the operation. It is implicitly differentiated from siblings like get_user_by_email and get_users, which use different lookup keys or return collections.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by ID' implies the tool is appropriate when a user_id is available, but the description gives no explicit guidance on when to prefer this over get_user_by_email or get_current_user. No exclusions or alternative routing is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_user_by_emailA

Find a user by email

ParametersJSON Schema
NameRequiredDescriptionDefault
emailYesUser email address

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries the full burden of behavioral disclosure. It only says 'Find a user by email' and does not explain what happens when no user matches, what fields are returned, whether the lookup is exact, or any error behavior. A read-only lookup is implied but not detailed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one short, direct sentence with no redundant words. All necessary information for a simple one-parameter tool is front-loaded, and every word contributes to meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one fully documented parameter, the description is adequate for basic selection. However, with no annotations and no output schema, the agent still lacks information about return values, error cases, and matching semantics, so completeness is only minimally acceptable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%: the email parameter is already described as 'User email address'. The description's 'by email' phrasing adds no significant semantic detail beyond the schema, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb ('Find'), a resource ('a user'), and a specific lookup method ('by email'). This distinguishes it from siblings like get_user and get_current_user, which access users through different identifiers or contexts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies this tool should be used when you have a user's email address, but it gives no explicit guidance on when not to use it or when to prefer alternatives like get_user, get_users, or get_current_user. The usage context is only inferred from the wording.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_usersC

List users

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNoMax items (max 250, default 250)
offsetNoOffset for pagination (default 0)
project_idNoFilter by project ID

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure, but it only says 'List users.' It does not mention pagination behavior, default limits, response format, filtering semantics beyond the schema, or any other runtime characteristics; the agent gets no behavioral insight beyond the action itself.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise with no filler or repetition; the two-word phrase earns its place. It is appropriately front-loaded, though it is so terse that it offers no additional structure beyond a bare statement of purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no annotations, no output schema, and three optional parameters, yet the description provides no context about return values, pagination, or when this tool is appropriate. While the schema documents parameters, the overall description is not complete enough for an agent to confidently select and invoke this tool without extra inference.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all three parameters (limit, offset, project_id) are already documented with meaningful descriptions. The tool description adds no parameter-level meaning, which is acceptable under the baseline for fully covered schemas.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the operation (List) and the resource (users), so the core purpose is unambiguous. However, it does not distinguish get_users from sibling tools such as get_user, get_current_user, or get_user_by_email, all of which also retrieve user information.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use get_users versus the several user-related sibling tools, nor any mention of exclusions or alternatives. The agent must infer from the plural resource name that this tool returns multiple users, but no explicit usage context is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

move_cases_to_sectionB

Move cases to another section

ParametersJSON Schema
NameRequiredDescriptionDefault
case_idsYesArray of case IDs to move
suite_idNoSuite ID
section_idYesSection ID

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full responsibility for behavioral disclosure. 'Move cases to another section' only states the basic action; it does not mention side effects like removal from the source section, possible data loss, permission requirements, or irreversibility.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, direct sentence with no filler. It front-loads the core operation and wastes no words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a mutating tool with no annotations and no output schema, yet the description is only a bare statement of intent. It does not explain what happens to the original location, whether suite_id is needed, or how this differs from copy_cases_to_section in practical use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema documents all parameters. The description adds only minimal clarification that section_id is the destination section ('another section'), but it does not explain the role of the optional suite_id or otherwise enrich the parameter semantics.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the exact operation: moving cases to a section. It clearly differentiates from the sibling tool 'copy_cases_to_section' by using 'move' rather than 'copy', and from 'move_section' by specifying 'cases' rather than 'section'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like copy_cases_to_section or move_section. The description implies the operation but does not state conditions, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

move_sectionB

Move a section to a new parent

ParametersJSON Schema
NameRequiredDescriptionDefault
after_idNoSection ID to insert after
parent_idNoNew parent section ID (null for root)
section_idYesSection ID

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description carries the full burden of disclosure. It only states the move operation and gives no detail about side effects, ordering behavior, handling of child sections, or whether the move is reversible. This is insufficient for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no extraneous words and is front-loaded with the action. It is appropriately brief for the operation, though it is minimal to the point of omitting useful context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With three parameters, including an optional ordering parameter and a nullable parent_id, the description does not explain the semantics of the move beyond the schema. There is no output schema and no annotation, so the agent lacks important behavioral context such as what happens to the section's children or how after_id is interpreted.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already describes all three parameters with 100% coverage. The description adds nothing about how after_id, parent_id, or section_id relate to the move operation beyond what the schema provides, so the baseline of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Move'), a resource ('a section'), and the destination ('a new parent'), making the operation clear. It is distinct from siblings like move_cases_to_section and update_section, though it doesn't explicitly name them.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to choose this tool over alternatives such as update_section or move_cases_to_section. The usage is only implied by the operation described, with no explicit context or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_caseC

Update a test case

ParametersJSON Schema
NameRequiredDescriptionDefault
refsNoComma-separated references
titleNoCase title
case_idYesTest case ID
type_idNoCase type ID
estimateNoEstimated time
priority_idNoPriority ID
template_idNoTemplate ID
milestone_idNoMilestone ID

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. 'Update a test case' only indicates mutation; it does not explain partial-update semantics, required fields beyond case_id, whether unspecified fields are left untouched, or any side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, short sentence with no wasted words, making it easy to parse. However, it is under-specified for an 8-parameter mutation tool; brevity here comes at the cost of useful context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the absence of annotations, no output schema, and eight parameters, the description is not complete enough. An agent needs to know whether this is a partial update, what happens to omitted fields, and how it differs from the bulk update sibling update_cases.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all parameters are already documented individually. The description adds no additional meaning about how parameters interact or which are required for a meaningful update, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Update') and the resource ('a test case'), which is specific enough for basic understanding. However, it does not distinguish this tool from the sibling 'update_cases', which likely handles bulk updates.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given about when to use this tool versus alternatives like add_case or update_cases. The singular 'a test case' implies it updates one case, but this is not explicit, and no conditions or exclusions are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_casesB

Bulk update multiple test cases

ParametersJSON Schema
NameRequiredDescriptionDefault
type_idNoCase type ID
case_idsYesArray of case IDs to update
suite_idNoSuite ID (required for multi-suite projects)
project_idYesProject ID
priority_idNoPriority ID
template_idNoTemplate ID
milestone_idNoMilestone ID

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral-disclosure burden. It only says 'update', which is already in the tool name, and adds no information about side effects, partial failures, required permissions, field-overwrite semantics, or response behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very concise and front-loaded, with no filler words. Every word contributes meaning. However, the brevity borders on under-specification for a tool of this complexity.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a bulk-mutation tool with seven parameters, no annotations, and no output schema, the description is too thin. It lacks guidance on multi-suite requirements, partial-update behavior, return values, and when to use this tool versus singular alternatives. The schema covers parameters but not the operational context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All seven parameters are documented in the input schema, so schema description coverage is 100%. The description itself adds no parameter-level meaning, so the baseline score of 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the verb ('update'), the resource ('test cases'), and the scope ('bulk', 'multiple'). This distinguishes it from the singular sibling 'update_case' and from delete/move operations, so an agent can tell which tool to select without opening the schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'bulk update multiple test cases' implies the tool is for updating more than one case at a time, but it never explicitly states when to use it over 'update_case' or what the exclusion criteria are. The usage guidance is only implied, not actionable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_projectC

Update an existing project

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoProject name
project_idYesProject ID
announcementNoProject description/announcement
is_completedNoMark project as completed
show_announcementNoShow announcement

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full burden of behavioral disclosure. It only indicates that an existing project will be updated, but does not disclose whether the update is partial or full replacement, what happens if the project_id does not exist, whether permissions are required, or whether changes are reversible.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single concise sentence with no redundancy or filler, and the core action is front-loaded. However, it is arguably too terse for a mutation tool with five parameters, so it earns a high but not perfect score for conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutation tool with no annotations and no output schema, this description is incomplete. It does not explain update semantics, error behavior, required parameters beyond what the schema lists, or how the tool relates to sibling tools. An agent would need to infer critical behavior from the tool name and schema alone.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% parameter description coverage, so the schema already documents all five parameters. The description adds no additional parameter semantics, but the baseline of 3 is appropriate since the schema handles this dimension adequately.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Update') and the resource ('an existing project'), which distinguishes it from sibling tools like get_project, add_project, and delete_project. However, it does not specify which project properties can be updated or any scoping details, so it is clear but not maximally specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No usage guidance is provided. The description does not explain when to use this tool versus add_project for creation, get_project for retrieval, or update_suite for other resource types. The word 'existing' weakly implies the project must already exist, but that is not explicit guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_runC

Update a test run

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoRun name
refsNoComma-separated references
run_idYesTest run ID
case_idsNoCase IDs
descriptionNoRun description
include_allNoInclude all test cases
milestone_idNoMilestone ID

TDQS

C2.4/5.0
Behavior1/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The tool has no annotations, so the description carries the full burden of behavioral disclosure. It only restates the operation name ('Update a test run') and provides no information about side effects, overwrite semantics, interactions between include_all and case_ids, or permission requirements. An agent gets no actionable behavioral context beyond the verb 'update'.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is terse but borders on under-specification rather than effective conciseness. 'Update a test run' conveys almost no more than the tool name itself and does not earn its place by adding scope, constraints, or context.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with seven parameters, no annotations, and no output schema, the description is far too minimal. It fails to communicate when updating a run is appropriate, what fields might conflict (e.g., include_all vs case_ids), or what the mutation entails. The schema documents the parameters but the description does not contextualize them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so all seven parameters already have descriptions in the input schema. The tool description adds no parameter-level meaning or usage nuance, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a clear verb ('Update') and resource ('test run'), which distinguishes it from sibling run operations like add_run, close_run, delete_run, and get_run. It does not enumerate which fields are updatable, but the schema provides that detail, so the purpose is sufficiently clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is given on when to use this tool versus alternatives such as add_run or close_run. The description does not state that the run must already exist, whether partial updates are supported, or when one would prefer update_run over other run-related tools. There are no exclusions or alternative hints.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_sectionC

Update a section

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoSection name
section_idYesSection ID
descriptionNoSection description

TDQS

C2.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of disclosing behavior, but it only conveys that some mutation occurs. It does not state whether fields are replaced wholesale, whether omitted fields are left unchanged, or what happens after the update.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three words is under-specification rather than genuine conciseness. The sentence is short but not front-loaded with useful information because it only restates the tool name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has no annotations and no output schema, and the description omits update semantics such as partial versus full replacement or whether section_id is the only required identifier. Although the schema fully documents parameters, the description leaves an agent guessing about the mutation behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and each parameter has a clear description, so the schema already documents name, section_id, and description. The description itself adds no parameter-level meaning, which is acceptable at the baseline for full schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose2/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description is a bare restatement of the tool name — 'Update a section' adds no information beyond the name itself. It does not say what aspects of a section can be updated or how this differs from add_section, move_section, or delete_section.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use update_section versus sibling tools such as add_section or move_section. The only implied usage is 'when a section already exists,' but no explicit conditions or exclusions are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_suiteC

Update a test suite

ParametersJSON Schema
NameRequiredDescriptionDefault
nameNoSuite name
suite_idYesSuite ID
descriptionNoSuite description

TDQS

C2.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It only says 'Update' and does not disclose whether updates are partial or full replacements, whether suite_id is immutable, what permissions are needed, or what the response looks like.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded, but its brevity reflects under-specification rather than deliberate economy. It conveys almost no information beyond what the tool name already provides.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a mutating tool with no annotations and no output schema, the description is incomplete. An agent cannot determine which fields may be changed, whether updates are additive or destructive to unspecified fields, or what a successful update returns. The schema helps with parameter names but not behavioral context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the parameters and their types are already documented. The description adds no semantic nuance beyond the schema, but it does not need to compensate for a coverage gap, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose3/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the verb 'Update' and resource 'test suite', so the basic operation is somewhat clear. However, it essentially restates the tool name and does not specify what aspects of the suite are updatable or how it differs from other update tools beyond the resource name.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus alternatives like add_suite, delete_suite, or update_project. The description provides no context about typical scenarios, prerequisites, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 54 tool updatesv1.0.0
    • First observedadd_case
    • First observedadd_case_field
    • First observedadd_project
    • First observedadd_result
    • First observedadd_result_for_case
    • First observedadd_results
    • First observedadd_results_for_cases
    • First observedadd_run
    • First observedadd_section
    • First observedadd_suite
    • First observedclose_run
    • First observedcopy_cases_to_section
    • First observeddelete_case
    • First observeddelete_cases
    • First observeddelete_project
    • First observeddelete_run
    • First observeddelete_section
    • First observeddelete_suite
    • First observedget_case
    • First observedget_case_fields
    • First observedget_case_statuses
    • First observedget_case_types
    • First observedget_cases
    • First observedget_current_user
    • First observedget_history_for_case
    • First observedget_priorities
    • First observedget_project
    • First observedget_projects
    • First observedget_result_fields
    • First observedget_results
    • First observedget_results_for_case
    • First observedget_results_for_run
    • First observedget_roles
    • First observedget_run
    • First observedget_runs
    • First observedget_section
    • First observedget_sections
    • First observedget_statuses
    • First observedget_suite
    • First observedget_suites
    • First observedget_templates
    • First observedget_test
    • First observedget_tests
    • First observedget_user
    • First observedget_user_by_email
    • First observedget_users
    • First observedmove_cases_to_section
    • First observedmove_section
    • First observedupdate_case
    • First observedupdate_cases
    • First observedupdate_project
    • First observedupdate_run
    • First observedupdate_section
    • First observedupdate_suite

TDQS

C2.9/5.0

Scored across 54 tools

Disambiguation3/5

The tools are mostly entity-based and distinct, but several groups can be confused: the four add_result variants differ mainly by test/case/bulk scope, and get_case vs get_test relies on the TestRail-specific distinction between a case definition and a test instance in a run. Descriptions help somewhat but do not fully disambiguate these boundaries.

Naming Consistency5/5

Tool names follow a highly consistent snake_case verb_noun pattern with predictable prefixes like get/add/update/delete and singular/plural conventions for object vs list operations. Special actions like close_run, move_section, and copy_cases_to_section still fit the same overall naming style.

Tool Count2/5

At 54 tools, this is far beyond the comfortable MCP range and will create a heavy selection surface for agents, even though each tool maps to a distinct TestRail API endpoint. The set is organized, but the sheer number makes it too large for efficient tool choice.

Completeness3/5

The core lifecycle for projects, suites, sections, cases, runs, and results is well covered, including bulk and move/copy operations. However, notable TestRail entities are missing—milestones, test plans, attachments, and configurations—so agents requiring those workflows will encounter dead ends.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    B
    quality
    D
    maintenance
    Enables AI assistants to interact directly with TestRail instances for managing test projects, suites, cases, runs, results, plans, milestones, and attachments through the TestRail API with secure authentication.
    77
    137 npm
    1
    MIT
  • A
    license
    A
    quality
    F
    maintenance
    Enables management of TestRail projects, test cases, runs, and results directly through MCP-supported clients. It provides a comprehensive set of tools to interact with the TestRail API for seamless test cycle management within AI environments.
    42
    1,101 npm
    44
    MIT