Skip to main content
Glama
alejandroviera

zephyr-squad-server-mcp

zephyr-squad-server-mcp

An MCP server that lets an AI agent drive Zephyr for Jira — Server / Data Center (ZAPI), the test-management suite for Jira. It exposes the most common test-management operations (cycles, folders, executions, test steps, step results, ZQL search) as well-described tools over stdio, so it plugs into Claude Desktop, Claude Code, and any other MCP client.

This targets the Server/DC flavor (/rest/zapi/latest). It is not for Zephyr Squad Cloud, which uses a different JWT-signed API.

Install

The server is published on PyPI and is easiest to run with uv:

uvx zephyr-squad-server-mcp        # run directly, no install
# or
uv tool install zephyr-squad-server-mcp
# or, in a venv
pip install zephyr-squad-server-mcp

Related MCP server: Zephyr Scale MCP Server

Configuration

The connection rides on your Jira instance's authentication (shared by ZAPI and the Jira core REST API). Configure via environment variables or a .env file (see .env.example):

Variable

Required

Default

Description

JIRA_URL

Jira base URL, e.g. https://jira.company.com (no /rest suffix).

JIRA_PERSONAL_TOKEN

one of

Personal Access Token → Authorization: Bearer … (recommended for DC).

JIRA_USERNAME + JIRA_PASSWORD

one of

Basic-auth fallback (password or API token). Used only if JIRA_PERSONAL_TOKEN is empty.

ZEPHYR_API_PATH

/rest/zapi/latest

ZAPI base path on the Jira host.

ZEPHYR_TIMEOUT

30

Per-request timeout (seconds).

ZEPHYR_VERIFY_SSL

true

Verify TLS certs. Set false only for trusted internal self-signed hosts.

The server fails fast with a clear error if the base URL or authentication is missing.

Client configuration (stdio)

Add to your MCP client config (e.g. Claude Desktop claude_desktop_config.json or Claude Code .mcp.json):

{
  "mcpServers": {
    "zephyr-squad": {
      "command": "uvx",
      "args": ["zephyr-squad-server-mcp"],
      "env": {
        "JIRA_URL": "https://jira.company.com",
        "JIRA_PERSONAL_TOKEN": "your-personal-access-token"
      }
    }
  }
}

For local development against a checkout, point command at uv:

{
  "mcpServers": {
    "zephyr-squad": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/zephyr-squad-server-mcp", "zephyr-squad-server-mcp"],
      "env": { "JIRA_URL": "https://jira.company.com", "JIRA_PERSONAL_TOKEN": "…" }
    }
  }
}

Tools

All id-bearing tools accept either a numeric id or a human-friendly key/name for project (key like SONY or its name), version (name like Version 1.0, or -1/Unscheduled), and issue (key like SONY-1386). Resolved ids are cached for the session, so the first call does the lookup and later calls reuse it.

Full reference: see docs/tools.md for every tool's arguments, types, defaults, the exact ZAPI endpoint it calls, and sync/async behavior. The table below is a quick index.

Tool

What it does

list_cycles

List test cycles for a project/version.

get_cycle

Get one cycle's details (id -1 = the Ad hoc cycle).

create_cycle

Create a test cycle.

update_cycle

Update a cycle's fields (id is sent in the body).

delete_cycle

Delete a cycle (async; auto-polls to completion by default).

export_cycle

Get a CSV download link for a cycle's executions.

copy_executions_to_cycle

Copy executions into a cycle (async; auto-polls to completion by default).

move_executions_to_folder

Move executions from a cycle into a folder (async; auto-polls to completion by default).

list_cycle_folders

List folders within a cycle.

create_folder

Create a folder under a cycle.

update_folder

Update a folder's name/description.

delete_folder

Delete a folder from a cycle (async; auto-polls to completion by default).

list_executions

List executions in a cycle (optionally a folder).

update_execution

Set an execution's status (PASS/FAIL/WIP/BLOCKED/…).

list_test_steps

List the test steps of a test issue.

create_test_step

Add a test step to a test issue.

list_step_results

List per-step results for an execution.

search_executions

Run a ZQL query and return matching executions.

get_execution_status_counts

Status-count rollup for a project/version (by cycle).

get_execution_status_counts_by_assignee

Per-assignee status counts for cycle(s).

list_executions_by_issue

A single test's executions across all cycles.

list_execution_defects

Defects linked to an execution.

get_tests_by_requirement

Tests covering requirement(s); flags orphans.

get_executions_by_test

Executions for a test (coverage history).

get_executions_by_defect

Executions linked to a defect.

get_defect_statistics

Per-defect execution/req/test rollups.

update_test_step

Edit a test step's action/data/result.

delete_test_step

Delete a test step.

create_execution

Schedule a test into a cycle.

add_tests_to_cycle_from_list

Bulk-add an explicit list of tests to a cycle as new UNEXECUTED executions (async).

add_tests_to_cycle_from_filter

Bulk-add tests matched by a saved ZQL filter to a cycle as new UNEXECUTED executions (async).

add_tests_to_cycle_from_cycle

Add tests from another cycle as new UNEXECUTED executions (async).

update_step_result

Set a step result's status/comment.

link_execution_defects

Link Jira defects to executions (async).

assign_execution

Assign an execution to a user.

delete_execution

Delete an execution by id.

get_job_progress (helper)

Manually poll any async job by token.

list_projects (helper)

List projects (name → id discovery).

list_versions (helper)

List a project's versions (name → id discovery).

get_zephyr_test_issue_type (helper)

Discover the Zephyr Test issue-type id per project (use case 1.1).

Dates use Jira's dd/MMM/yy format (e.g. 4/Dec/12). Execution-status codes: -1=UNEXECUTED, 1=PASS, 2=FAIL, 3=WIP, 4=BLOCKED, 5=PENDING, 6=APPROVED, 7=CANCELLED.

Development

uv sync                  # create venv + install deps (incl. dev group)
uv run pytest            # unit tests (httpx mocked with respx — no live Jira needed)
uv run ruff check        # lint
uv run mcp dev src/zephyr_squad_server_mcp/server.py   # MCP Inspector smoke test

docs/getzephyr.apib is the ZAPI API Blueprint used as the reference for request/response shapes during development. It was retrieved from the official public documentation at https://getzephyr.docs.apiary.io/ (© SmartBear Software). An older version is also available in the zfjdeveloper/zapi-docs repository.

The Inspector (mcp dev) launches the MCP Inspector via npx, so it needs Node.js / npx on your PATH. The tool list loads without any Jira config, but invoking a tool builds the client on first use — so set JIRA_URL and auth (a .env is read automatically) before running tools.

Publishing to PyPI

uv build                                  # produces wheel + sdist in dist/
uv publish --token "$UV_PUBLISH_TOKEN"    # or set UV_PUBLISH_TOKEN in the env

Test against TestPyPI first:

uv publish --publish-url https://test.pypi.org/legacy/ --token "$TESTPYPI_TOKEN"

License

MIT — see LICENSE.

Available Tools

40 tools
add_tests_to_cycle_from_cycleA

Add tests from another cycle as new UNEXECUTED executions in this cycle.

Tests are sourced from from_cycle_id / from_version_id and can be narrowed by components, labels, priorities, statuses (comma-separated filter strings) and has_defects (bool). project/version accept keys/names or ids (prefer ids to skip a lookup; -1 = Unscheduled). Async: with wait=True (default) polls to completion and returns the final job payload; with wait=False returns {jobProgressToken}.

Important: new executions always start with status UNEXECUTED regardless of the source cycle's results. If you need to carry over the original execution statuses and defect links, use copy_executions_to_cycle instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
projectYes
versionYes
from_cycle_idYes
from_version_idYes
componentsNo
labelsNo
prioritiesNo
statusesNo
has_defectsNo
assignee_typeNo
folder_idNo
waitNo
timeoutNo

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description covers key behaviors: new executions always start UNEXECUTED, async behavior with wait, project/version accept keys/names or ids (-1 = Unscheduled). It lacks details on error handling or rate limits but is still thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured with paragraphs and bullet points. Each sentence adds value without redundancy. It is concise yet comprehensive.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 14 parameters and no output schema, the description covers usage comprehensively. It explains filters, async mode, and key constraints. A minor gap is the lack of explanation for what 'jobProgressToken' entails, but overall complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but the description explains all parameters: source, filters (comma-separated strings, boolean), assignee_type, folder_id, wait, timeout. It adds meaning beyond raw schema by clarifying project/version id/name acceptance.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool adds tests from another cycle as new UNEXECUTED executions in this cycle. It specifies the source parameters (from_cycle_id/from_version_id) and distinguishes from the sibling tool copy_executions_to_cycle.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly tells when to use this tool vs. the alternative copy_executions_to_cycle (for carrying over original statuses and defects). It also explains async behavior with wait parameter.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_tests_to_cycle_from_filterB

Add tests matching a saved ZQL filter to a cycle as new UNEXECUTED executions.

search_id is the numeric id of a saved ZQL filter. project/version accept keys/names or ids (prefer ids to skip a lookup; -1 = Unscheduled). Async: with wait=True (default) polls to completion and returns the final job payload; with wait=False returns {jobProgressToken}. New executions always start with status UNEXECUTED.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
projectYes
versionYes
search_idYes
assignee_typeNo
folder_idNo
waitNo
timeoutNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description adds some behavioral context about async polling and UNEXECUTED status, but misses traits like required permissions, side effects, or return format details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with a single paragraph, front-loads the main purpose, and adds details without extraneous content; slight improvement could be achieved with bullet points.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters, async job operation, and no output schema, the description explains async behavior and execution status but lacks explanation for key schema fields and return values, and does not differentiate adequately among many sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, and the description only explains search_id, project, version, wait, and timeout. It omits meaning for cycle_id (required), assignee_type, and folder_id, leaving significant gaps.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Add' and the resource 'tests matching a saved ZQL filter to a cycle as new UNEXECUTED executions,' distinguishing it from sibling tools like add_tests_to_cycle_from_cycle and add_tests_to_cycle_from_list.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context about using IDs vs names and async behavior but does not explicitly state when to use this tool over alternatives, leaving usage comparison implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

add_tests_to_cycle_from_listA

Add an explicit list of tests to a cycle as new UNEXECUTED executions.

issues is a list of Zephyr Test issue keys/ids (e.g. ["PROJ-1", "PROJ-2"]); a comma-separated string is also accepted and split automatically. project/version accept keys/names or ids (prefer ids to skip a lookup; -1 = Unscheduled). Async: with wait=True (default) polls to completion and returns the final job payload; with wait=False returns {jobProgressToken}. New executions always start with status UNEXECUTED. To carry over existing execution statuses and defect links from another cycle, use copy_executions_to_cycle instead.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
projectYes
versionYes
issuesYes
assignee_typeNo
folder_idNo
waitNo
timeoutNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It discloses new executions start as UNEXECUTED, async polling with wait, and that 'issues' can be list or comma-separated. It does not mention rate limits or auth, but is fairly transparent.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the core action. It uses bullet points and examples effectively, though slightly lengthy. It could be more concise but is well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters and no output schema, the description covers the main behavior, async feature, and key sibling differentiation. It does not describe the final payload but is sufficient for an experienced user.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but description adds meaning for 'issues' format and 'wait' behavior. It mentions project/version accept keys/names or ids. However, it does not describe cycle_id, assignee_type, folder_id, timeout beyond defaults.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool adds an explicit list of tests to a cycle as new UNEXECUTED executions. It uses specific verb 'add' and resource 'tests to cycle', and distinguishes from sibling 'copy_executions_to_cycle'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on when to use this tool vs copy_executions_to_cycle (to carry over statuses). It explains the async behavior with wait parameter but does not explicitly state when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

assign_executionA

Assign an execution to a user.

assignee is a Jira username/account id. (execution_id is the schedule id from list_executions.) Returns the API's (empty) acknowledgement.

ParametersJSON Schema
NameRequiredDescriptionDefault
execution_idYes
assigneeYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses return value (empty acknowledgement) and implies mutation, but without annotations it could cover permissions or side effects more thoroughly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two efficient sentences with no redundancy; front-loads main action and provides parameter context concisely.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Sufficient for a simple assignment tool given sibling context; mentions return type but could include error handling or constraints.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds essential meaning beyond schema: clarifies assignee format and execution_id source, compensating for 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Assign an execution to a user' with specific verb and resource. It distinguishes from siblings by focusing on assignment, not creation or update.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides guidance on parameter sources (Jira username, schedule ID from list_executions), but lacks explicit when-to-use or alternatives compared to similar tools like assign_execution vs. update_execution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

copy_executions_to_cycleA

Copy executions into cycle cycle_id, preserving their original status and defect links.

Unlike add_tests_to_cycle_from_cycle, this tool carries over the execution status (PASS, FAIL, etc.) and defect associations from the source executions. Set clear_status_flag=True to reset statuses to UNEXECUTED, or clear_defect_mapping_flag=True to drop defect links on copy. If you want fresh UNEXECUTED executions without any prior results, prefer add_tests_to_cycle_from_cycle instead. project/version accept keys/names or ids (prefer ids to skip a lookup; -1 = Unscheduled). Async: with wait=True (default) polls to completion and returns the final job payload; with wait=False returns {jobProgressToken}.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
executionsYes
projectYes
versionYes
clear_status_flagNo
clear_defect_mapping_flagNo
waitNo
timeoutNo

TDQS

A4.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses copying behavior, optional flags to reset statuses/defects, async polling with wait parameter, and input type flexibility. Lacks explicit mention of permissions or side effects, but covers key behaviors well given no annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Efficient and well-structured. Front-loaded with purpose, then contrast, parameter specifics, and async behavior. No wasted sentences, clear bullet-point style.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers most aspects: purpose, flags, async, parameter usage. Minor gap: doesn't explicitly state that executions are IDs, but schema and context imply it. Missing detailed output for non-wait case, but sufficient for typical use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

All 8 parameters are semantically explained: cycle_id, executions, project/version (with id/name note), clear_status_flag, clear_defect_mapping_flag, wait, and timeout. Adds meaning beyond the schema names.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it copies executions into a cycle, preserving status and defect links, and distinguishes itself from the sibling tool `add_tests_to_cycle_from_cycle` by highlighting the difference in preserving prior results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states when to use this tool vs. the alternative (`add_tests_to_cycle_from_cycle`) and provides guidance on async behavior (wait flag) and parameter preferences (prefer ids for project/version).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_cycleA

Create a new test cycle.

Dates use Jira's dd/MMM/yy format (e.g. 4/Dec/12). project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup). The new cycle is created in version — pass it explicitly (use -1 only if you really want the Unscheduled version). Returns {id, responseMessage}.

ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
projectYes
versionYes
buildNo
environmentNo
descriptionNo
start_dateNo
end_dateNo
sprint_idNo
cloned_cycle_idNo

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full disclosure burden. It reveals date format requirements, parameter input flexibility (keys/names vs ids), and return format. It lacks details on auth, rate limits, or potential side effects, but for a create operation, these are acceptable gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, front-loaded with the core action, and uses minimal sentences to convey essential details. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 10 parameters and no output schema, the description covers key behavioral aspects (date format, parameter types, return structure) but omits details for optional parameters. It is sufficient for a create tool but could be more thorough.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must compensate. It adds meaning for project, version (prefer ids), and date format, but does not explain build, environment, description, start_date, end_date, sprint_id, or cloned_cycle_id. Partial coverage warrants a mid score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Create a new test cycle' with a specific verb and resource. It adds concrete details like date format and parameter options, making the tool's purpose unmistakable and distinguished from sibling tools like add_tests_to_cycle_from_cycle or update_cycle.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides guidance on when to use the explicit version and cautions against using -1 unless intentionally targeting the Unscheduled version. However, it does not explicitly list alternatives or state when not to use this tool, leaving some ambiguity.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_executionA

Schedule a test (create an execution) in a cycle.

issue is the Zephyr test issue (key or id; prefer the id when you have it). cycle_id is the target cycle (-1 = Ad hoc). project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). Optionally set folder_id and assignee. Returns a map keyed by the new execution id.

ParametersJSON Schema
NameRequiredDescriptionDefault
issueYes
cycle_idYes
projectYes
versionYes
folder_idNo
assigneeNo
assignee_typeNo

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that `project`/`version` accept keys/names or ids with preference for ids to skip lookups, and explains special values like `-1` for `cycle_id` and `version`. It also mentions the return format (a map keyed by execution id). However, it does not discuss permissions, rate limits, or potential errors, leaving some gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with about 5 sentences, front-loading the main purpose. Every sentence adds value: purpose, parameter details with special values, optional fields, and return type. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 7 parameters, no output schema, and no annotations, the description covers purpose, parameter semantics, and output format well. It lacks usage guidelines and error scenarios, but for a creation tool it is sufficiently complete to enable correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides no parameter descriptions (0% coverage). The tool's description compensates fully by explaining each parameter's meaning, acceptable types (key/id, name/id), special values (`-1`), and optionality (`folder_id`, `assignee`). This gives the agent rich context beyond raw schema types.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Schedule a test (create an execution) in a cycle.' This provides a specific verb (Schedule) and resource (execution). However, it does not explicitly distinguish from sibling tools like `add_tests_to_cycle_from_cycle` which may also create executions, so it misses the highest clarity mark.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives such as `add_tests_to_cycle_from_list` or `update_execution`. The description focuses solely on how to use parameters, omitting context for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_folderA

Create a folder under a cycle.

project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). Returns {id, responseMessage}.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
nameYes
projectYes
versionYes
descriptionNo

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the return format ('{id, responseMessage}') and explains how 'project' and 'version' parameters are interpreted. It does not mention permissions, error scenarios, or constraints like unique folder names, but it is adequate for a simple creation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is extremely concise: two sentences, front-loading the main purpose and then providing parameter hints and return type. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 5 parameters (4 required), no output schema, and no annotations, the description covers the essential aspects: purpose, return format, and parameter nuances for key fields. It lacks details on the 'description' parameter and constraints like uniqueness, but it is largely complete for a straightforward folder creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds meaning for 'project' and 'version' (accept keys/names/ids, prefer ids, -1 = Unscheduled). However, it does not explain 'cycle_id', 'name', or 'description' beyond their names and types in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Create a folder') and the context ('under a cycle'). This distinguishes it from sibling tools like 'create_cycle' (creates cycles) and 'update_folder' (updates folders).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context (folder creation under a cycle) and gives usage hints for the 'project' and 'version' parameters (prefer ids, use -1 for Unscheduled). However, it does not explicitly state when to use this tool versus alternatives or mention any prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_stepA

Add a test step to a Zephyr test issue.

issue accepts a Jira issue key or numeric issueId (prefer the id when you have it, to skip a lookup). step is the action, data the test data, result the expected result. Returns the created step.

ParametersJSON Schema
NameRequiredDescriptionDefault
issueYes
stepYes
dataNo
resultNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries burden. It explains creation and return of step, but doesn't disclose ordering, limits, or side effects like whether steps are appended or inserted.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with front-loaded purpose. No wasted words; every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers all 4 parameters, return value, and core behavior. Missing validation details like max lengths or constraints, but adequate for a simple creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema coverage, description compensates well: explains issue accepts key or numeric ID, step is action, data is test data, result is expected result. Adds useful semantic detail beyond schema titles.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Add' and resource 'test step to a Zephyr test issue', with specific parameter roles. It distinguishes from siblings like delete_test_step and update_test_step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides guidance on parameter usage (prefer issue ID, step/data/result meaning) but no explicit when-to-use or alternatives among many sibling tools. Lacks context for tool selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_cycleA

Delete a test cycle by id (asynchronous; returns/polls a job token).

Set is_folder_cycle_delete=True only when deleting a cycle that lives inside a folder. This is an asynchronous ZAPI operation: it returns a jobProgressToken. With wait=True (default) the tool polls the job to completion and returns the final job-progress payload; with wait=False it returns the raw {jobProgressToken} so you can poll later with get_job_progress (type cycle_delete_job_progress).

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
is_folder_cycle_deleteNo
waitNo
timeoutNo

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description explains the asynchronous operation, return of a job token, and the two modes of waiting. It details the polling behavior and references get_job_progress for manual polling. Given no annotations are provided, the description adequately covers the behavioral traits beyond the delete action, though it does not mention error handling or side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused paragraph that front-loads the main action and async behavior. It is informative without being verbose, though it could benefit from slight restructuring (e.g., bullet points) for easier parsing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a deletion tool with no output schema and 4 parameters, the description provides good context: async behavior, job token details, wait modes, and reference to a related tool. It lacks explicit return value format or error scenarios, but covers the essential usage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description compensates by explaining three of four parameters: cycle_id (implicit), is_folder_cycle_delete (when to use), and wait (synchronous vs. asynchronous). Timeout is mentioned with a default but not explicitly described; however, its role in polling is inferable from context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Delete a test cycle by id'. It specifies the asynchronous nature and distinguishes the operation from sibling tools like delete_execution or delete_folder by mentioning the folder cycle delete flag. The verb+resource combination is specific and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to set is_folder_cycle_delete and explains the wait parameter behavior for synchronous vs. asynchronous usage. However, it does not explicitly state when not to use this tool (e.g., for deleting folders) or mention prerequisites like required permissions, which would enhance decision-making.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_executionA

Delete an execution by its numeric id.

Args: execution_id: numeric id of the execution to delete.

Returns: {"success": "Successfully deleted execution(s) "}

ParametersJSON Schema
NameRequiredDescriptionDefault
execution_idYes

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It indicates a destructive action but does not disclose if deletion is reversible, what happens to related data, or any side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Very concise with two sentences and clear Args/Returns structure. No extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description is adequate but lacks details on response or side effects. Could mention that deletion is permanent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so the description must explain parameters. It describes execution_id as 'numeric id of the execution to delete', adding meaning beyond the schema's type and title.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Delete an execution by its numeric id', which is a specific verb and resource. It distinguishes from sibling tools like create_execution, update_execution, and list_executions by focusing on deletion.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for deleting executions but does not explicitly state when to use or avoid this tool. No mention of prerequisites or conditions for safe use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_folderA

Delete a folder from a cycle (asynchronous; returns/polls a job token).

folder_id is the numeric ID from list_cycle_folders; cycle_id is its parent cycle. project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). With wait=True (default) the tool polls the job to completion and returns the final job-progress payload; with wait=False it returns the raw {jobProgressToken} to poll later with get_job_progress.

ParametersJSON Schema
NameRequiredDescriptionDefault
folder_idYes
cycle_idYes
projectYes
versionYes
waitNo
timeoutNo

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses asynchronous behavior, job token return, and polling detail. Since no annotations are provided, the description carries the full burden; it lacks info on cascading deletes, permissions required, or error handling, but the core behavior is well covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (about 5 lines) and front-loaded with the main purpose. It uses parentheses and semicolons to pack additional information without verbosity. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description explains the return values (job token or final payload). It mentions the polling mechanism and references get_job_progress for follow-up. However, it does not describe the job-progress payload structure or error states.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description explains 5 of 6 parameters: folder_id, cycle_id, project/version (with usage hints), and wait. It omits timeout, but overall adds significant meaning beyond the schema, such as noting that folder_id comes from list_cycle_folders.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Delete a folder from a cycle' with a specific verb and resource. It distinguishes from sibling tools like create_folder, update_folder, and delete_cycle by focusing on folder deletion. The asynchronous nature is front-loaded.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides explicit usage guidance: it explains the wait parameter with two modes (wait=True polls to completion, wait=False returns a token for later polling via get_job_progress). It also gives tips on project/version parameter values (prefer ids, -1 for Unscheduled).

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_test_stepB

Delete a test step. Returns the remaining steps for the issue.

ParametersJSON Schema
NameRequiredDescriptionDefault
issueYes
step_idYes

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses that the tool deletes a step and returns remaining steps, which is useful. However, it omits behavioral details like irreversibility, permission requirements, or side effects on related objects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with two sentences, no redundancy. It is front-loaded with the action. However, it could be slightly more structured (e.g., separating purpose and output).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool, the description provides a basic understanding of purpose and return value. However, it lacks parameter details and usage context, which are needed given no output schema and minimal annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, and the description adds no explanation for parameters. It does not clarify what 'issue' refers to (key, ID?) or the expected format of 'step_id'. The schema provides titles but no descriptions, and the description fails to compensate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action ('Delete') and the resource ('a test step'), and distinguishes from sibling tools like create_test_step, update_test_step, and other delete tools. Also mentions that it returns remaining steps, providing additional clarity on the outcome.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives (e.g., delete_cycle, delete_execution). It does not specify prerequisites, such as requiring issue and step_id to exist, or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

export_cycleA

Get a download link for a cycle's executions as a CSV export.

project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). Pass folder_id to export a single folder within the cycle (omit for the whole cycle). Returns {url: "…/Cycle-RC1.csv"} — a download link, not the file bytes; hand the URL to the user or fetch it separately.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
projectYes
versionYes
folder_idNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description discloses that it returns a download link (not file bytes) and the response format. It adds details on parameter semantics. It could mention that it is a read-only operation, but overall transparency is good.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise and well-structured. It leads with the main purpose, then provides parameter details and return value in a clear, scannable format. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 4 parameters, no output schema, and no annotations, the description covers the essential aspects: parameter flexibility, folder export option, and the nature of the return value. It could mention error handling or what the URL should be used for, but overall it is fairly complete for a straightforward export tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but the description adds meaning: it explains that project/version accept various types and -1 for Unscheduled, and folder_id is optional for partial export. cycle_id is not elaborated, but the description compensates well for the lack of schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Get a download link for a cycle's executions as a CSV export.' It specifies the resource (cycle's executions) and action (download link), distinguishing it from sibling tools like get_cycle or list_executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context: it explains parameter types ('project/version accept keys/names or ids'), preference for ids, and the optional folder_id. It also clarifies what the tool returns. However, it does not explicitly state when not to use this tool or compare to alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_cycleA

Get a single test cycle's details by its numeric id.

Passing -1 returns the hardcoded Ad hoc cycle.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes

TDQS

A3.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It only mentions parameter behavior and special value -1, but omits return structure (what 'details' are), error handling, or idempotency. For a read operation, more context on safety and response is needed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no filler. First sentence states core purpose, second adds critical special case. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Adequate for a simple get tool given one required parameter, but lacks output structure, error scenarios, and explicit read-only guarantee. With no output schema or annotations, additional detail on return values would improve completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but description adds significant value by explaining 'numeric id' and noting the special '-1' value for Ad hoc cycle, which is not evident from the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states verb ('Get') and resource ('a single test cycle's details') with specificity ('by its numeric id'). Includes a special case note about '-1' for Ad hoc cycle, distinguishing itself from siblings like list_cycles or update_cycle.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies use when a specific cycle id is known, but no explicit guidance on when to use alternatives (e.g., list_cycles for listing, or update_cycle for modifications). No when-not or prerequisite conditions mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_defect_statisticsA

Get execution/requirement/test statistics for one or more defects.

defects is a Jira key/id or a comma-separated string / list of them. Returns per-defect rollups (execution status totals, requirement & test counts) — useful for defect-density and closure-rate narratives.

ParametersJSON Schema
NameRequiredDescriptionDefault
defectsYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes return value and input format, but with no annotations, it doesn't explicitly state read-only nature or other behavioral traits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Very concise, three sentences, front-loaded with main purpose, no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers input and output adequately for a single-parameter tool without output schema, though could specify exact return field names.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds value beyond schema by explaining defects parameter as Jira key/id and acceptable formats (comma-separated string or list), compensating for 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Explicitly states it gets statistics for defects, listing types (execution, requirement, test) and mentioning per-defect rollups, distinguishing from siblings like get_executions_by_defect.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides context of use (defect-density and closure-rate narratives) but does not explicitly compare to or exclude alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_executions_by_defectA

List executions linked to a given defect (incl. step-level defects).

defect is a Jira bug key or id. Returns {executions, totalCount}.

ParametersJSON Schema
NameRequiredDescriptionDefault
defectYes
max_recordsNo
offsetNo

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It states it lists executions with step-level defects and the return format. However, it does not disclose pagination behavior, rate limits, or read-only nature beyond what is inferred. Adequate but minimal extra info beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences plus a note; every sentence provides value. No fluff. Front-loaded with the primary purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 3 parameters, no output schema, and no annotations, the description covers the main purpose, defect parameter, and return shape. However, it could be more complete on pagination details (e.g., max_records limits) and error cases. Still mostly adequate for a simple list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so description compensates partially. It explains the defect parameter as a Jira bug key or id, and mentions return shape. But it does not describe max_records or offset beyond schema defaults. Adds meaning for one parameter, not all.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists executions linked to a given defect, including step-level defects. It specifies the verb 'list' and resource 'executions', and distinguishes from siblings like get_executions_by_test or list_executions by focusing on defect linkage.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage when a defect is known (defect is a Jira bug key or id), but does not explicitly exclude or compare to related tools like link_execution_defects or list_execution_defects. No when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_executions_by_testA

List executions for a given test issue (its execution history & defects).

test is a Jira test issue key or id. Use to flag tests that have never been executed (audit coverage).

ParametersJSON Schema
NameRequiredDescriptionDefault
testYes
max_recordsNo
offsetNo

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It describes listing executions and defects but omits details on pagination (offset/max_records), sorting, limits, or read-only nature. Parameter explanation for 'test' is helpful, but overall behavioral traits are underdisclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose and immediate context. No wasted words. Efficiently communicates core information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 3 parameters (one explained), no output schema, no annotations, and many siblings, the description is adequate but incomplete. Lacks pagination behavior, output format, error conditions, or differentiation from similar tools like 'list_executions_by_issue'.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 0% coverage. Description explains the 'test' parameter (Jira issue key or id), adding meaning. However, 'max_records' and 'offset' are left unexploited; their defaults are not mentioned. With low coverage, partial compensation yields a 3.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the tool lists executions for a given test issue, including history and defects. The verb 'List' and resource 'executions' are specific. It adds a use case for auditing coverage. However, it does not explicitly differentiate from sibling 'list_executions_by_issue'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit usage context: 'Use to flag tests that have never been executed (audit coverage).' This gives a clear scenario. However, it lacks guidance on when not to use or alternatives among siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_execution_status_countsA

Status-count rollup for a project/version (PASS/FAIL/WIP/BLOCKED/UNEXECUTED).

With cycles set (comma-separated cycle ids, e.g. "51,52") returns the status breakdown for those cycles. Without cycles, returns a per-cycle breakdown for every cycle in the version. project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). Useful for release Go/No-Go and compliance summaries.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectYes
versionYes
cyclesNo
foldersNo
componentsNo
offsetNo
limitNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses behavioral traits: the effect of the cycles parameter, that project/version accept keys/names or ids, and that -1 = Unscheduled. It does not mention pagination behavior or rate limits, but the core read-only behavior is implied. Overall, it provides good transparency beyond the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, using two sentences plus a brief note on id usage and a use case. It is front-loaded with the main purpose and quickly covers parameter behavior. There is no redundancy, though a more structured layout could improve readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 7 parameters (2 required) and no output schema, the description covers the main modes (with/without cycles), explains id vs key usage, and gives a use case. It does not explain folders, components, or the exact shape of the output, but these are either self-explanatory or secondary. Overall, it provides sufficient context for an agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0% (only param titles provided). The description adds meaning for cycles (comma-separated), project/version (keys/names/ids, -1), and hints at filtering. However, it does not explain folders, components, offset, limit, or output format. This partial coverage is adequate but not thorough.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it provides a status-count rollup for a project/version, enumerating statuses like PASS, FAIL, WIP, BLOCKED, UNEXECUTED. It distinguishes from siblings like get_execution_status_counts_by_assignee by focusing on overall rollup. Use cases (release Go/No-Go, compliance summaries) further clarify purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains when to use the cycles parameter vs not (with cycles returns breakdown for those cycles, without cycles returns per-cycle breakdown). It advises preferring ids when available to skip lookups. It does not explicitly contrast with sibling tools like get_executions, but the context is clear enough for typical usage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_execution_status_counts_by_assigneeB

Per-assignee status counts for the given cycle(s).

cycles is a comma-separated list of cycle ids. project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). Returns a map of assignee -> {UNEXECUTED, PASS, FAIL, WIP, BLOCKED}. Use to spot workload imbalance or reviewers/testers who are a bottleneck.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectYes
versionYes
cyclesNo
offsetNo
limitNo

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description must disclose behavior. It explains parameter formats (cycles comma-separated, project/version accept keys/names/ids, -1 for Unscheduled) and the return map structure. However, offset and limit parameters are not explained, and there is no mention of authentication, rate limits, or error cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with two main sentences and inline parameter notes. It avoids unnecessary verbosity while conveying key information. The structure is functional, though the inline notes could be slightly clearer in a separate section.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (5 parameters, no output schema, no annotations), the description covers the main return format and parameter quirks, but lacks explanation for offset/limit, error handling, and prerequisites (e.g., project existence). It provides enough for basic usage but leaves gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must add meaning. It clarifies the cycles format and the flexible input types for project/version. However, offset and limit are left unexplained, and the default value for cycles (empty string) is not discussed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns per-assignee status counts for given cycles, and specifies the output as a map with status fields. The name and phrasing distinguish it from the sibling 'get_execution_status_counts' but does not explicitly state the difference.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides one concrete use case ('spot workload imbalance or bottlenecks'), but does not specify when to avoid using it or mention alternative tools like 'get_execution_status_counts' for aggregate counts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_job_progressA

Poll an asynchronous ZAPI job once by its progress token.

type identifies the job kind, e.g. bulk_execution_copy_move_job_progress (copy/move executions) or cycle_delete_job_progress. Returns the current progress payload (progress reaches 1.0 when complete; message holds the result).

ParametersJSON Schema
NameRequiredDescriptionDefault
tokenYes
typeYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It explains the one-time poll nature and progress indicator (progress reaches 1.0 when complete, message holds result). However, it omits error handling, authentication needs, and rate limits, leaving some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the main action, no unnecessary words. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, but description covers key return aspects (progress value up to 1.0, message field). Missing full structure and error cases, but sufficient for a simple polling tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so description must compensate. It explains 'type' with examples of job kinds, adding context beyond schema. The 'token' parameter is only named, lacking explanation of its origin or format, which is a minor gap.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool polls an asynchronous ZAPI job by its progress token, with specific verb 'poll' and resource 'job progress'. It distinguishes from sibling tools that focus on cycles, executions, or folders, making its unique purpose evident.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for polling job progress but does not explicitly state when to use or not use this tool versus alternatives. No guidance on prerequisites or scenarios where polling is inappropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_tests_by_requirementA

Find the Zephyr tests that cover one or more requirement issues.

requirements is a Jira issue key/id, or a comma-separated string / list of them (e.g. "PROJ-1,PROJ-2" or ["PROJ-1", "PROJ-2"]). Returns, per requirement, the linked tests and a defect count — use to flag orphaned requirements (empty tests) and to map AC coverage.

ParametersJSON Schema
NameRequiredDescriptionDefault
requirementsYes

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It mentions returns 'linked tests and a defect count per requirement' and use cases, but lacks details on pagination, project scope, or read-only nature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with a clear front-loaded purpose statement. Every sentence adds value: purpose, parameter format, return summary, and use cases. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (1 parameter, no output schema, no annotations), the description covers the core purpose and parameter adequately. It lacks detailed return format but is sufficient for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, but description significantly adds meaning: it specifies parameter format ('Jira issue key/id, comma-separated string, or list') and provides examples. This compensates fully for missing schema details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Find' and the resource 'Zephyr tests that cover requirement issues'. It distinguishes itself from siblings by focusing on requirements coverage, while siblings handle cycles, executions, steps, etc.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear usage context: 'use to flag orphaned requirements (empty `tests`) and to map AC coverage.' It implicitly indicates when to use but does not explicitly mention alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_zephyr_test_issue_typeA

Discover the Zephyr "Test" issue-type id per project (for use case 1.1).

Returns {options: [{label, type, value}, ...]} where label is the project key/name and value is that project's Zephyr Test issue-type id. Use this to find the issue-type id the Jira/Atlassian MCP needs when creating Zephyr Test issues (Zephyr does not create Jira issues itself).

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It states the return format and meaning of fields, and explains that this ID is needed for creation. It is clearly a read operation, and no side effects are mentioned. It adds useful context beyond just the input schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with 4 sentences, each serving a purpose: purpose, return format, usage, and context. No waste, well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no parameters and no output schema, the description covers the essential aspects: what it returns, how to use it, and why. It could mention how project keys are obtained, but it's still fairly complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are 0 parameters, so schema coverage is 100%. The description adds value by explaining the return format and usage, meeting the baseline of 4 for no parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool discovers the Zephyr 'Test' issue-type id per project, explicitly distinguishing it from sibling tools that deal with cycles, executions, etc. The verb 'Discover' and resource 'issue-type id' are specific.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use this tool when needing the issue-type id for creating Zephyr Test issues, and notes that Zephyr does not create Jira issues itself. It provides clear context but does not list exclusions, though they may not be needed given the sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_cycle_foldersA

List the folders within a cycle.

project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). Returns a list of folders, each with ID, folderName, folderDescription, and execution summaries.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
projectYes
versionYes
limitNo
offsetNo

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that project/version accept names or ids, prefers ids, and that -1 means Unscheduled. Specifies return fields: ID, folderName, folderDescription, and execution summaries. Lacks explanation of pagination (limit/offset) behavior, but overall provides good context for a list operation without annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states core purpose, second provides parameter and return details. No redundancy, every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers main behavior and return structure, but omits pagination behavior, error scenarios, and ordering. Given no output schema, description explains return fields, but gaps remain for a list tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds meaning for project and version parameters (accepts names/ids, special value). Does not explain cycle_id, limit, or offset beyond schema types. With 0% schema description coverage, this partial compensation is adequate but not complete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List the folders within a cycle' with a specific verb and resource. It distinguishes from siblings like list_cycles and list_executions. Additional details about parameter handling and return structure reinforce purpose.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. The description hints at input handling (prefer ids, -1 for Unscheduled) but does not clarify when to choose this over other list tools or mention any prerequisites or constraints.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_cyclesA

List test cycles for a project and version.

project accepts a Jira project key (e.g. SONY), a project name, or a numeric projectId. version accepts a version name, a numeric versionId, or -1/Unscheduled. Prefer the numeric ids when you have them, to skip a lookup. Returns a map keyed by versionId; each cycle entry is keyed by cycleId (the Ad hoc cycle is -1).

ParametersJSON Schema
NameRequiredDescriptionDefault
projectYes
versionYes
offsetNo
expandNo

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes return format (map keyed by versionId, cycle entries keyed by cycleId, Ad hoc cycle -1). No annotations exist, so description carries full burden. Does not explicitly state it's read-only, but the action 'List' implies non-destructive behavior. Missing details on auth or rate limits, but acceptable for a list operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two paragraphs: first sentence states purpose, second details parameters. Efficient and front-loaded. No redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers core functionality and return structure. However, lacks details on pagination (offset) and the 'expand' parameter. For a list tool, these are relevant for using the tool correctly. Without an output schema, description should cover more behavioral context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%, so description must add meaning. It explains the two required parameters (project, version) well, including accepted types and preference for IDs. However, it does not explain the optional 'offset' and 'expand' parameters, leaving them undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the tool lists test cycles given a project and version. The verb 'List' and resource 'test cycles' are specific. However, it does not explicitly differentiate from sibling list tools like list_cycle_folders or list_executions, though the resource is distinct.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides concrete guidance on parameter values: project accepts key/name/numericId, version accepts name/id/-1, and advises preferring numeric IDs to skip lookups. Does not address when to use this tool vs alternatives like search_executions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_execution_defectsB

List Jira defects currently linked to an execution.

execution_id is the numeric id from list_executions/search_executions. Returns a map keyed by execution id -> {defectKey -> defect summary}.

ParametersJSON Schema
NameRequiredDescriptionDefault
execution_idYes

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses the return format but does not mention permissions, side effects, error conditions, or pagination. As a read operation, it is likely safe, but the lack of behavioral details is a gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with three sentences, front-loading the purpose. The return format description is a bit informal but clear. No wasted words, but could be more structured (e.g., bullet points).

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple nature of the tool (one parameter, no output schema), the description covers the core functionality, parameter source, and return format. However, it lacks details on error handling and the exact structure of the returned map (e.g., fields of defect summary).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description compensates by specifying that execution_id is a numeric id from list_executions/search_executions. This adds meaningful context beyond the schema's type definition, though more details (e.g., valid range) could be included.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists Jira defects linked to an execution, which is a specific verb-resource pair. It distinguishes from siblings like link_execution_defects (modification) and get_defect_statistics (aggregates). However, the return format mentioning 'map keyed by execution id' introduces slight ambiguity since only one execution_id is input.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions that execution_id comes from list_executions/search_executions, providing source context. It does not explicitly state when to use this tool versus alternatives like link_execution_defects or get_defect_statistics, but the read vs. write distinction is implied.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_executionsA

List the executions in a cycle (optionally narrowed to a folder).

project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). Returns {status, executions, recordsCount} where status maps status ids to their definitions.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
projectYes
versionYes
folder_idNo
offsetNo
limitNo

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses return structure ({status, executions, recordsCount}) and parameter behavior (project/version accept keys/names/ids, -1 for Unscheduled). But does not explain pagination (offset, limit) or side effects; no annotations provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences: purpose, parameter tips, return shape. No unnecessary words, well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose, key parameters, and return shape. Missing pagination details and explanation of cycle_id. No differentiation from search_executions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0%. Description adds meaning for project/version (keys/names/ids, special -1) and folder_id (narrowing). But cycle_id, offset, limit are not explained beyond schema defaults.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List the executions in a cycle (optionally narrowed to a folder)', specifying the verb (list), resource (executions), and scope (cycle, optionally folder). This distinguishes it from siblings like search_executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Implies usage context: for listing executions within a cycle, optionally narrowed by folder. Advises preferring IDs for project/version. However, no explicit when-not or alternatives among sibling tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_executions_by_issueB

List all executions of a single test issue, across every cycle.

issue accepts a Jira issue key (e.g. SONY-1386) or numeric id — passed straight through (the endpoint accepts either). Returns the test's full execution history, useful for flaky-test diagnosis and coverage checks.

ParametersJSON Schema
NameRequiredDescriptionDefault
issueYes
actionNo
offsetNo
max_recordsNo
expandNo

TDQS

B3.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that the tool returns executions 'across every cycle' and explains the issue parameter format. However, it does not cover pagination (offset/max_records), the 'expand' parameter, or behavior when the issue is not found. Since no annotations exist, the description carries the burden but is incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (two sentences), front-loaded with the core purpose, and includes a helpful example. No wasted words, though a bit more structure could improve readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, no annotations, and 5 parameters with only one explained, the description lacks completeness. It does not mention return format, pagination, or the effect of 'expand' and 'action', leaving significant gaps for an agent to use the tool correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It only explains the 'issue' parameter format, leaving 'action', 'offset', 'max_records', and 'expand' undocumented. This is insufficient for a 5-parameter tool.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists all executions for a single test issue, using a specific verb ('List') and resource ('executions of a single test issue'). It also mentions the parameter format. However, it does not explicitly differentiate from siblings like 'list_executions' or 'search_executions', which also list executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides a clear use case ('flaky-test diagnosis and coverage checks'), implying when to use the tool. However, it lacks explicit guidance on when not to use it or how it compares to siblings such as 'search_executions' or 'get_executions_by_test'.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_projectsA

List all Jira projects visible to Zephyr (name/key -> projectId).

Returns {options: [{label, value, type}, ...]} where label is the project name and value is its numeric id. Use this to discover ids; most tools also accept a project key/name directly.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. It discloses that the tool returns a list of options with label, value, and type, explaining what each field means. It indicates the operation is a read-only list (no side effects). Could mention pagination or ordering but not necessary for a simple list.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose, followed by output format and usage guidance. No unnecessary words. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no parameters, no output schema, no annotations), the description fully covers what the agent needs: what the tool does, what it returns, and when to use it. Nothing missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Tool has zero parameters. Score baseline is 4. Description adds value by explaining the output format and usage, going beyond the schema. It effectively makes the tool self-explanatory despite no parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'List all Jira projects visible to Zephyr' with a specific verb and resource, and distinguishes from sibling tools that focus on cycles, executions, etc. The mapping from name/key to projectId is explicitly mentioned.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit usage advice: 'Use this to discover ids; most tools also accept a project key/name directly.' This tells when to use the tool (to get numeric ids) and implies when not needed (if key/name works directly). No explicit exclusion of alternatives, but clear context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_step_resultsA

List the per-step results for an execution.

execution_id is the numeric id of an execution (from list_executions or search_executions). Returns the step-result records, each with a status code (-1=UNEXECUTED, 1=PASS, 2=FAIL, 3=WIP, 4=BLOCKED).

ParametersJSON Schema
NameRequiredDescriptionDefault
execution_idYes
expandNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavior. It explains the return format including status codes, which is helpful, but does not mention read-only nature, pagination, or other side effects. Adequate but not comprehensive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is efficient, using two sentences to convey purpose, parameter sourcing, and return format. There is no wasted text, and critical information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description includes return field details (status codes with meanings), which is good. It does not mention optional fields, but for a list tool, the provided information is reasonably complete. The lack of annotations is compensated by the clear explanation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must add meaning. For 'execution_id', it provides useful context (numeric id from specific tools). However, 'expand' is unexplained, leaving a gap. The description partially compensates for the lack of schema docs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List the per-step results for an execution,' using a specific verb and resource. This differentiates it from sibling tools like 'list_test_steps' which lists test steps, not step results.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context by noting that 'execution_id' comes from 'list_executions' or 'search_executions', guiding the agent on where to obtain the required parameter. It lacks explicit exclusions or alternative tool references but is sufficient for effective use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_test_stepsB

List the test steps of a Zephyr test issue.

issue accepts a Jira issue key (e.g. SONY-1386) or a numeric issueId (prefer the id when you have it, to skip a lookup). Returns {stepBeanCollection: [...]}.

ParametersJSON Schema
NameRequiredDescriptionDefault
issueYes
offsetNo
limitNo

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must carry full burden. Only mentions return format as stepBeanCollection but does not disclose authentication needs, error handling, or that it only works for Zephyr test issues.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Minimal and efficient: two sentences and a return type line. Front-loaded with purpose, then parameter guidance, then output format. No redundant text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, so description must detail return values. Only a brief mention of stepBeanCollection without field specs. Missing pagination behavior, default limit, and any error conditions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema coverage, description adds meaning only for 'issue' (key or id preference). Parameters offset and limit have no description, relying on name inference. Incomplete compensation for missing schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it lists test steps of a Zephyr test issue using the verb 'List' and resource 'test steps'. It distinguishes from sibling tools like list_step_results by focusing on steps for an issue.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides guidance on the issue parameter (prefer id to skip lookup) but lacks context on when to use this tool vs alternatives like list_step_results. No mention of pagination usage or conditions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_versionsA

List the versions of a project (name -> versionId).

project accepts a key, name, or numeric id (prefer the id when you have it, to skip a lookup). Returns {unreleasedVersions, releasedVersions} where each entry has label (version name) and value (versionId). -1 is the Unscheduled version.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectYes

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so the description bears full weight. It explains the output structure ({unreleasedVersions, releasedVersions} with label and value) and notes that -1 represents Unscheduled version. It does not mention error cases or destructive effects, but as a read operation, this is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise: a single-line purpose, followed by parameter details and output format. Every sentence adds value, and the structure is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema), the description covers all necessary aspects: parameter usage, output format, and special version constant. No gaps remain.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema only specifies project as string or integer. The description adds crucial meaning: it accepts key, name, or numeric id, and advises preferring id to skip a lookup. This fully compensates for the 0% schema coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'List the versions of a project' with a mapping of name to versionId. It is specific about the resource and action, and distinguishes itself from sibling tools that deal with cycles and executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides explicit guidance on how to use the project parameter (key, name, or id, preferring id), but does not discuss when to use this tool versus alternatives. Since no sibling tool lists versions, this is acceptable.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

move_executions_to_folderA

Move executions from a cycle into an internal folder.

Moves all executions in the cycle when schedules is omitted, or only the listed execution ids when schedules is provided. project/version accept keys/names or ids (prefer ids when you have them, to skip a lookup; -1 = Unscheduled). This is an asynchronous ZAPI operation: it returns a jobProgressToken. With wait=True (default) the tool polls the job to completion and returns the final job-progress payload; with wait=False it returns the raw {jobProgressToken} so you can poll later with get_job_progress.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
folder_idYes
projectYes
versionYes
schedulesNo
waitNo
timeoutNo

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses the asynchronous nature, return of jobProgressToken, and polling behavior. However, it does not detail side effects (e.g., whether executions are removed from original location), permission requirements, or error scenarios.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is well-structured: a brief purpose sentence followed by detailed behavior. It front-loads the core action, then explains conditional logic and async behavior concisely without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description adequately covers return values (jobProgressToken or final payload) and key parameters. It lacks error handling details but is sufficient for understanding the tool's functionality.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Since schema coverage is 0%, the description compensates well. It explains the dual behavior of schedules (null vs list), the project/version parameter types with preference for ids, and the wait/timeout parameters. Only the timeout unit is implicit (likely seconds), but overall adds significant meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool moves executions from a cycle into an internal folder. It distinguishes between moving all executions when schedules is omitted or only specific ones when provided, which sets it apart from sibling tools like copy_executions_to_cycle or create_folder.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear guidance on when to use the schedules parameter and explains the async behavior with wait=True/False. However, it does not explicitly state when not to use this tool or compare it to alternatives, though the context makes it clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_executionsC

Run a ZQL query and return matching executions.

Example ZQL: project = "SONY" AND executionStatus = FAIL. Returns {executions, totalCount, ...}.

ParametersJSON Schema
NameRequiredDescriptionDefault
zql_queryYes
offsetNo
max_recordsNo
expandNo

TDQS

C2.7/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description carries full burden. Only states it returns executions and totalCount; no disclosure of side effects, authorization needs, rate limits, or pagination behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Description is very concise with two sentences and an example. Front-loads the main action. Could include brief parameter descriptions without much bloat, but current structure is efficient.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness1/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, no parameter descriptions, and no annotations, the description is highly incomplete. Fails to provide enough context for an agent to invoke the tool correctly, especially for parameters like expand.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%. Description does not explain any of the four parameters (zql_query, offset, max_records, expand). Only an example ZQL query is given, leaving agent uninformed about other parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the tool runs a ZQL query and returns matching executions. Provides an example query and return structure hint. Distinguishes from siblings by specifying ZQL as the filtering mechanism.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus siblings like list_executions or get_executions_by_defect. The description implies usage for ZQL queries but does not specify alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_cycleA

Update an existing test cycle's fields.

The cycle id is sent in the body (PUT /cycle, no id in the path) — only the fields you pass are included; omit a field to leave it unchanged. Dates use Jira's dd/MMM/yy format (e.g. 8/Aug/14). To change the version pass version (a name or id); a version name also needs project to resolve it (a numeric version needs neither). Returns {error, success, noPermission}.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycle_idYes
nameNo
buildNo
environmentNo
descriptionNo
start_dateNo
end_dateNo
versionNo
folder_idNo
projectNo

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description takes full responsibility. It discloses the HTTP method (PUT), body-only id, partial update behavior, date format, version resolution dependency, and return format. Missing are permissions, side effects, or lock details, but core behavioral traits are well covered.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact: one sentence for purpose, then three concise points about mechanics, and a final line on return value. Zero wasted words, front-loaded, and uses formatting (bold, code) effectively.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 10 parameters, no output schema, and no annotations, the description covers purpose, update behavior, date format, version logic, and return format. It lacks explanation of field meanings (e.g., build, environment), potential side effects, or prerequisites. Adequate but incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It adds critical semantics: cycle_id sent in body, omit fields to leave unchanged, date format, version as name/id with project requirement. However, not every parameter is individually explained (e.g., build, environment, folder_id rely on general 'omit' rule).

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description starts with a clear verb+resource: 'Update an existing test cycle's fields.' This immediately distinguishes it from create_cycle, delete_cycle, and other cycle-related siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

While the description provides detailed parameter usage (e.g., date format, version resolution), it does not explicitly state when to use this tool versus alternatives like update_execution or create_cycle. No comparison or 'when not to use' guidance is given.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_executionA

Set an execution's status by its numeric id.

status is a ZAPI execution-status code: -1=UNEXECUTED, 1=PASS, 2=FAIL, 3=WIP, 4=BLOCKED, 5=PENDING, 6=APPROVED, 7=CANCELLED. Returns the updated execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
execution_idYes
statusYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the return value ('Returns the updated execution'), but does not mention side effects, error cases, or idempotency. For a mutation tool, this is a moderate gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the purpose, and contains no filler. Every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simplicity of the tool, the description covers the essential information: purpose, status codes, and return. However, it could be improved by noting constraints (e.g., only updates status, not assignment) and error handling.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% coverage, so the description must compensate. It explains the status parameter's possible values well, but does not clarify the execution_id beyond 'numeric id.' Partially compensates for status but not for id.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action: 'Set an execution's status by its numeric id.' It specifies the verb (set/update), resource (execution status), and mechanism (numeric id), effectively distinguishing it from sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context by listing valid status codes, helping the agent understand when to use this tool. However, it lacks explicit guidance on when not to use it or alternatives for other operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_folderA

Update a folder's name and/or description.

The API requires the folder's full context, so cycle_id and project/version (which accept keys/names or ids — prefer ids when you have them, to skip a lookup; -1 = Unscheduled) must be supplied alongside folder_id, the numeric ID from list_cycle_folders. Only the optional fields you pass are sent; omit one to leave it unchanged. Returns {id, responseMessage}.

ParametersJSON Schema
NameRequiredDescriptionDefault
folder_idYes
cycle_idYes
projectYes
versionYes
nameNo
descriptionNo

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided; description discloses partial update behavior (only passed fields updated) and return format {id, responseMessage}. Adds value by explaining the API's requirement for full context.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Concise and well-structured: starts with main action, then requirements, optional fields, and return format. No wasted words; every sentence adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers main functionality, required params, optional update behavior, and return format. Lacks edge cases but is complete for a simple update tool given no output schema or annotations.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but description explains the role of each required parameter, including that folder_id comes from list_cycle_folders, and that project/version accept keys/names or IDs. Adds meaning beyond schema types and titles.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'Update a folder's name and/or description', specifying the exact verb and resource, and differentiating from create/delete siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context on required parameters (folder_id, cycle_id, project/version) and explains that only passed optional fields are updated, with a tip to prefer IDs. Does not explicitly state when not to use, but context is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_step_resultA

Set the status (and optional comment) of one step result.

Get step_result_id from list_step_results. status codes: -1=UNEXECUTED, 1=PASS, 2=FAIL, 3=WIP, 4=BLOCKED. Step results are created automatically when an execution is opened, so there is no separate create.

ParametersJSON Schema
NameRequiredDescriptionDefault
step_result_idYes
statusYes
commentNo

TDQS

A4.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It explains that the tool sets status and comment, and that step results are auto-created. It does not disclose return values, error conditions, or permission requirements, leaving some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences plus a list of status codes. The action is front-loaded, and every sentence adds value without redundancy. Perfectly concise.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple update tool, the description covers main usage points. However, the lack of output schema means the description should ideally mention what the tool returns (e.g., success indicator), which is absent. Still, it is largely complete given the tool's simplicity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage, so the description adds all parameter meaning. It explains step_result_id source, status codes with explicit mapping, and that comment is optional. This fully compensates for the lack of schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action 'Set the status (and optional comment) of one step result,' using a specific verb and resource. It is distinct from sibling tools which focus on other entities like cycles or executions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It provides helpful context: where to obtain the step_result_id, the meaning of status codes, and that step results are auto-created. However, it does not explicitly compare with alternatives or state when not to use this tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_stepA

Update fields of an existing test step.

Only the provided fields are sent (omit to leave unchanged). step is the action, data the test data, result the expected result. issue accepts a key or id (prefer the id when you have it, to skip a lookup). Returns the updated step.

ParametersJSON Schema
NameRequiredDescriptionDefault
issueYes
step_idYes
stepNo
dataNo
resultNo

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Covers partial update and return value, but with no annotations, description should disclose more behavioral traits like idempotency or side effects. Adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Concise, front-loaded with purpose, then details. Every sentence adds value with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers partial update, field meanings, and return value. No output schema, but description suffices. Could explicitly state required parameters, but schema already indicates them.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds meaning beyond schema by describing each parameter's purpose (action, test data, expected result) and provides guidance for 'issue' to prefer id. Schema coverage is 0%, so description compensates well.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states 'Update fields of an existing test step', with verb and resource. Distinguishes from sibling tools like create_test_step and delete_test_step.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explains partial update behavior ('only the provided fields are sent') and clarifies the role of each field. No explicit when-not-to-use or alternatives, but context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 40 tool updatesv0.1.0
    • First observedadd_tests_to_cycle_from_cycle
    • First observedadd_tests_to_cycle_from_filter
    • First observedadd_tests_to_cycle_from_list
    • First observedassign_execution
    • First observedcopy_executions_to_cycle
    • First observedcreate_cycle
    • First observedcreate_execution
    • First observedcreate_folder
    • First observedcreate_test_step
    • First observeddelete_cycle
    • First observeddelete_execution
    • First observeddelete_folder
    • First observeddelete_test_step
    • First observedexport_cycle
    • First observedget_cycle
    • First observedget_defect_statistics
    • First observedget_execution_status_counts
    • First observedget_execution_status_counts_by_assignee
    • First observedget_executions_by_defect
    • First observedget_executions_by_test
    • First observedget_job_progress
    • First observedget_tests_by_requirement
    • First observedget_zephyr_test_issue_type
    • First observedlink_execution_defects
    • First observedlist_cycle_folders
    • First observedlist_cycles
    • First observedlist_execution_defects
    • First observedlist_executions
    • First observedlist_executions_by_issue
    • First observedlist_projects
    • First observedlist_step_results
    • First observedlist_test_steps
    • First observedlist_versions
    • First observedmove_executions_to_folder
    • First observedsearch_executions
    • First observedupdate_cycle
    • First observedupdate_execution
    • First observedupdate_folder
    • First observedupdate_step_result
    • First observedupdate_test_step

TDQS

A3.7/5.0

Scored across 40 tools

Disambiguation5/5

Every tool has a clearly distinct purpose. Even similar tools like add_tests_to_cycle_from_cycle and copy_executions_to_cycle have explicit descriptions differentiating their behavior (UNEXECUTED vs. preserving status). Agents should reliably select the correct tool.

Naming Consistency5/5

All tools follow a consistent verb_noun pattern in snake_case. Verbs like list_, create_, update_, delete_, add_, copy_, move_ are appropriately chosen. Even longer names like get_execution_status_counts_by_assignee maintain consistency.

Tool Count3/5

40 tools is heavy for a single server, with many specialized operations (e.g., multiple ways to add tests). While the domain justifies many tools, the count is borderline excessive and may overwhelm agents. A more curated set (20-25) would improve focus.

Completeness4/5

The tool set covers CRUD for cycles, folders, executions, test steps, plus search, status reports, defect linking, and async job polling. Minor gaps exist: no tool to unlink defects from executions, and test steps cannot be reordered. Overall, core workflows are well supported.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers