Skip to main content
Glama

QTM4J MCP Server

npm npm downloads Node License

An MCP server with 87 tools for QMetry Test Management for Jira (QTM4J). Search and manage test cases, cycles, executions, plans, folders, comments, defects, automation rules, and project metadata from Claude Desktop, Claude Code, VS Code Copilot, Cursor, or any MCP-compatible client.

Distribution:

Quick start (no clone required)

You need a QMetry API key (QMetry → API Keys) and Node.js 18+.

Claude Desktop

Edit your config file and restart Claude:

  • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

  • Windows: %APPDATA%\Claude\claude_desktop_config.json

  • Linux: ~/.config/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "qtm4j": {
      "command": "npx",
      "args": ["-y", "qtm4j-mcp-server@^0.1"],
      "env": {
        "QTM4J_API_KEY": "your-api-key-here",
        "QTM4J_REGION": "US"
      }
    }
  }
}

Claude Code (CLI)

claude mcp add qtm4j -e QTM4J_API_KEY=your-api-key-here -e QTM4J_REGION=US -- npx -y qtm4j-mcp-server@^0.1

VS Code (GitHub Copilot Agent mode)

Create .vscode/mcp.json in your workspace:

{
  "servers": {
    "qtm4j": {
      "type": "stdio",
      "command": "npx",
      "args": ["-y", "qtm4j-mcp-server@^0.1"],
      "env": {
        "QTM4J_API_KEY": "${env:QTM4J_API_KEY}",
        "QTM4J_REGION": "US"
      }
    }
  }
}

Switch Copilot Chat to Agent mode and the qtm4j_* tools appear automatically.

Cursor

Add to ~/.cursor/mcp.json (or <project>/.cursor/mcp.json for project-level):

{
  "mcpServers": {
    "qtm4j": {
      "command": "npx",
      "args": ["-y", "qtm4j-mcp-server@^0.1"],
      "env": { "QTM4J_API_KEY": "your-api-key-here" }
    }
  }
}

The example configs above pin the package to ^0.1 so a future breaking release won't auto-upgrade you. Drop the @^0.1 suffix if you'd rather always run the latest.

Global install (faster startup)

npx re-resolves the package on every launch, which adds a few seconds of startup latency. If you use the server frequently, install it globally and point your client at the binary directly:

npm install -g qtm4j-mcp-server

Then in your client config, replace the npx command:

{
  "mcpServers": {
    "qtm4j": {
      "command": "qtm4j-mcp-server",
      "env": {
        "QTM4J_API_KEY": "your-api-key-here",
        "QTM4J_REGION": "US"
      }
    }
  }
}

Tradeoff: instant startup and works offline, but you'll need to run npm update -g qtm4j-mcp-server to get new versions.

Related MCP server: qtm4j-mcp-server

Configuration

Variable

Required

Default

Description

QTM4J_API_KEY

yes

QMetry API key, sent on every request as the apiKey header

QTM4J_REGION

no

US

UShttps://qtmcloud.qmetry.com, AUhttps://syd-qtmcloud.qmetry.com

💡 Set QTM4J_REGION=AU if your QMetry instance is on the Sydney cluster.

Tools

All tools are prefixed with qtm4j_ to avoid collisions with other MCP servers.

Area

Tools

Test Cases

qtm4j_create_test_case, qtm4j_get_test_case, qtm4j_search_test_cases, qtm4j_update_test_case, qtm4j_delete_test_case, qtm4j_archive_test_case, qtm4j_unarchive_test_case, qtm4j_clone_test_cases, qtm4j_create_test_steps, qtm4j_update_test_steps

Test Cycles

qtm4j_create_test_cycle, qtm4j_get_test_cycle, qtm4j_search_test_cycles, qtm4j_update_test_cycle, qtm4j_delete_test_cycle, qtm4j_archive_test_cycle, qtm4j_unarchive_test_cycle

Test Executions

qtm4j_get_test_cycle_executions, qtm4j_get_test_execution, qtm4j_update_test_execution, qtm4j_update_test_step_execution, qtm4j_bulk_update_test_executions, qtm4j_get_execution_teststeps, qtm4j_update_execution_custom_fields, qtm4j_upload_execution_attachment, qtm4j_list_execution_attachments, qtm4j_delete_execution_attachment

Test Plans

qtm4j_create_test_plan, qtm4j_get_test_plan, qtm4j_search_test_plans, qtm4j_update_test_plan, qtm4j_delete_test_plan, qtm4j_archive_test_plan, qtm4j_unarchive_test_plan, qtm4j_link_test_cycles_to_plan, qtm4j_get_linked_test_cycles, qtm4j_unlink_test_cycles_from_plan

Comments

qtm4j_get_test_case_comments, qtm4j_add_test_case_comment, qtm4j_update_test_case_comment, qtm4j_delete_test_case_comment, qtm4j_get_test_cycle_comments, qtm4j_add_test_cycle_comment, qtm4j_update_test_cycle_comment, qtm4j_delete_test_cycle_comment, qtm4j_get_test_plan_comments, qtm4j_add_test_plan_comment, qtm4j_update_test_plan_comment, qtm4j_delete_test_plan_comment

Defects

qtm4j_get_execution_defects, qtm4j_link_execution_defects, qtm4j_unlink_execution_defects, qtm4j_get_step_execution_defects, qtm4j_link_step_execution_defects, qtm4j_unlink_step_execution_defects, qtm4j_search_cycle_defects, qtm4j_get_cycle_defect_summary

Folders

qtm4j_list_folders, qtm4j_create_folder

Automation

qtm4j_link_automation_rule, qtm4j_unlink_automation_rule, qtm4j_run_automation_rules

Metadata (read-only)

qtm4j_get_projects, qtm4j_get_priorities, qtm4j_get_priority_icons, qtm4j_get_statuses, qtm4j_get_environments, qtm4j_get_builds, qtm4j_get_labels, qtm4j_get_components, qtm4j_get_execution_results, qtm4j_get_custom_fields, qtm4j_get_parameters, qtm4j_get_user_permissions

See docs/TOOLS.md for the full reference and docs/COOKBOOK.md for example prompts colleagues can paste into any MCP client.

Every tool ships with annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) so clients can decide whether to ask for confirmation. Read tools accept a response_format parameter (json default, or markdown for human-readable output). Large responses are automatically truncated at 25k characters with a hint to narrow the query.

All tools validate inputs with Zod, paginate via startAt/maxResults, and automatically retry rate-limited (HTTP 429) responses with exponential back-off (up to 3 attempts).

Trying it out

Once connected, ask your assistant something like:

Search QMetry project <your project ID> for test cases with status "To Do" and show me the first 5.

The client will call qtm4j_search_test_cases and render the response.

Get all executions in test cycle <KEY>-TR-747 and mark any unexecuted ones as Pass.

Example tool calls

Replace <projectId> and <KEY> with your tenant's numeric Jira project ID and project key. Find them in the project URL: …?projectId=<projectId>&projectKey=<KEY>. Use qtm4j_get_execution_results to discover your tenant's executionResultId values.

// Search test cases
{
  "name": "qtm4j_search_test_cases",
  "arguments": {
    "projectId": "<projectId>",
    "status": ["Approved"],
    "maxResults": 20,
    "response_format": "markdown"
  }
}

// Update an execution result
{
  "name": "qtm4j_update_test_execution",
  "arguments": {
    "cycleId": "<internal-cycle-id>",
    "testCaseExecutionId": "<internal-tc-execution-id>",
    "executionResultId": "<pass-id>",
    "comment": "Verified on staging"
  }
}

Troubleshooting

  • Tools don't appear in my client. Restart the client after editing config. Check claude mcp list (Claude Code) or VS Code's MCP panel for connection status. On first run, npx -y qtm4j-mcp-server may take a few seconds to download the package.

  • 401 Unauthorized. Your QTM4J_API_KEY is invalid or expired. Generate a new one in QMetry → API Keys.

  • 404 on execution or search endpoints. Many endpoints want the internal numeric id, not the human key like <KEY>-TR-747. Call qtm4j_get_test_cycle first to translate the key into the internal id.

  • Empty or oversized folder response. Pass folderId to qtm4j_list_folders to scope to a subtree — full project trees on large projects can exceed the response size limit.

  • projectId rejected. Use the numeric Jira project ID, not the project key string. You can find it in the Jira project URL: …?projectId=<numeric-id>&projectKey=<KEY>.

Notes

  • Search endpoints use POST /…/search — filters go in the body under filter, pagination/sort on the query string. Tool handlers wrap this for you.

  • 204 No Content responses resolve as { message: "…" }.

  • The Swagger spec does not currently document a framework-style automation import-result endpoint (e.g. JUnit/TestNG/Cucumber ingestion); the automation tools cover the rules-run and rule-link flows exposed in the spec.

Development

Local setup if you want to modify the server:

git clone https://github.com/salehrifai42/qmetrymcp.git
cd qmetrymcp
npm install
npm run build
QTM4J_API_KEY=your-key npm start

Test changes with the MCP Inspector:

QTM4J_API_KEY=your-key npx @modelcontextprotocol/inspector node dist/index.js

Run the unit tests (no API key or network needed — the HTTP client's transport is faked in-memory):

npm test

Replicating the bulk xlsx import workflow

The repo ships a Claude Code skill (.claude/skills/xlsx-to-qmetry/) and a Python importer (scripts/import-xlsx-to-qmetry.py) for pushing folders of Excel test cases into QMetry. Tenant-specific IDs (project, parent folder, status, custom-field IDs, components) are kept out of git — you supply them in your own config.json.

First-run after cloning:

cp config.template.json config.json
# Edit config.json — fill in:
#   connection.apiKey         (QMetry → avatar → API Keys → Generate)
#   connection.projectId      (numeric Jira project ID)
#   xlsxImport.parentFolderId (target folder for new test cases)
#   xlsxImport.statusId       (e.g. Draft / Approved)
#   xlsxImport.apiTestFieldId (custom field ID for "API Test" toggle, if used)
#   xlsxImport.options        (Yes/No option IDs for that custom field)
#   xlsxImport.componentIds   (default components to attach; can be [])

cp .claude/commands/qtm4j.template.md .claude/commands/qtm4j.md
# Optional: populate qtm4j.md with your tenant's IDs using the GET endpoints listed inside.

Discover the IDs you need with the running MCP server (qtm4j_get_projects, qtm4j_get_statuses, qtm4j_get_components, qtm4j_get_custom_fields, qtm4j_list_folders) or with node scripts/refresh-field-reference.mjs to dump everything to field_reference.json.

Then launch Claude Code from the repo root and the xlsx-to-qmetry skill auto-loads. Drop your workbooks under Input/<batch-name>/ and ask Claude to import them — see docs/EXCEL-IMPORT-GUIDE.md for the full workflow.

Bugs and contributions

Found a bug or want to suggest a feature? Open an issue at https://github.com/salehrifai42/qmetrymcp/issues. PRs welcome.

License

MIT

Available Tools

30 tools
bulk_update_test_executionsA

Apply the same execution result, environment, or build to multiple test case executions at once. Use testCycleTestCaseMapIds from get_test_cycle_executions (the 'testCycleTestCaseMapId' field). Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycleIdYesTest cycle ID
testCycleTestCaseMapIdsYesTest-case-execution map IDs to update
executionResultIdNoExecution result ID to apply to all
environmentIdNoEnvironment ID to apply to all
buildIdNoBuild ID to apply to all

TDQS

A4.1/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description discloses the return code (204) but omits behavioral details such as partial failure handling, idempotency, permissions, or the optional nature of the result/environment/build parameters.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with three sentences: purpose, usage hint, and return code. It is front-loaded and contains no extraneous information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's complexity and lack of output schema, the description could provide more context on batch size limits, atomicity, and error handling to be fully complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for each parameter. The description adds value by explaining the source for testCycleTestCaseMapIds and clarifying the purpose of the update fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool applies the same execution result, environment, or build to multiple test case executions at once, distinguishing it from single-update siblings like update_test_execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description advises using testCycleTestCaseMapIds from get_test_cycle_executions, providing a specific data source. However, it lacks explicit guidance on when to use this tool versus alternatives and does not mention prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

clone_test_casesA

Bulk clone one or more test cases into a target project and optional folder. Returns a background task object with a taskId and progressUrl to poll for completion.

ParametersJSON Schema
NameRequiredDescriptionDefault
testcaseIdsYesTest case IDs to clone
projectIdYesTarget project ID or key
folderIdNoTarget folder ID

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must disclose behavior. It mentions returning a background task with taskId and progressUrl, but does not clarify side effects (e.g., whether source remains), authentication needs, or rate limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states action and target, second describes return value. Front-loaded with key information, no superfluous words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers purpose and return format adequately. Lacks details on scope of cloning (e.g., attachments, steps) and limits, but is reasonably complete for a simple tool without output schema.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description restates the parameter roles without adding extra meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states action (clone), resource (test cases), and destination (target project and optional folder). It distinguishes from sibling tools like create_test_case or delete_test_case by specifying 'bulk clone'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like create_test_case or search_test_cases. Usage is implied by the word 'clone' but lacks direct exclusions or comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_folderA

Create a new folder under an existing parent folder. Use parentId=0 for root-level. folderName is required. Returns the created folder with its id. Use list_folders first to find valid parentId values.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
folderNameYesFolder name
folderTypeYesFolder type
parentIdYesParent folder ID (use 0 for root)
descriptionNoFolder description

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Describes return value ('Returns the created folder with its id') and implies creation behavior. No annotation provided, so description carries full burden; lacks error condition details but adequate for a creation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with verb and resource. Every sentence adds value; no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 5 parameters, no output schema, and sibling tools, the description covers purpose, parameter usage, return, and prerequisite tool. Thorough and actionable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds meaning beyond schema: clarifies parentId=0 for root and pre-requisite of list_folders. Schema coverage is 100% but description adds actionable context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states action ('Create a new folder') and resource ('under an existing parent folder'). Distinguishes from sibling 'list_folders' by recommending it as a precursor.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly guides on using parentId=0 for root, requiring folderName, and advising to use list_folders first to find valid parentId values.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_caseB

Create a new test case in QMetry. Returns the created test case object including its internal id and key (e.g. FS-TC-123). Priority, status, labels, and components use integer IDs — see field_reference.json for valid values.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
summaryYesTest case title/summary
preconditionNoPrecondition / description text
priorityNoPriority integer ID (e.g. 600784 for High)
statusNoStatus integer ID (e.g. 544256 for Done)
assigneeNoAssignee Jira account ID
labelsNoLabel IDs to attach
componentsNoComponent IDs
fixVersionsNoFix version IDs
folderIdNoTarget folder ID
customFieldsNoCustom field values

TDQS

B3.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations exist, so description must disclose behavioral traits. It only mentions creation and return value, omitting side effects, permission requirements, or rate limits. Minimal transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no fluff. Purpose, return value, and key parameter hint are front-loaded. Every sentence is meaningful.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 11 parameters and no output schema, description could be more complete. It explains integer-ID fields and return value, but leaves out customFields, folderId, and other parameter semantics. Adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverate is 100%, but description adds value by summarizing that priority, status, labels, components use integer IDs and referencing field_reference.json for valid values. This helps agents beyond the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it creates a test case in QMetry and returns the created object with id and key. However, it does not explicitly differentiate from sibling tools like clone_test_cases or update_test_case.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides guidance that integer IDs come from field_reference.json, but no advice on when to use this vs alternatives (e.g., cloning or importing). No context on prerequisites or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_cycleA

Create a new test cycle in QMetry. Returns the created cycle with its internal id and key (e.g. FS-TR-123). Use list_folders with folderType=TESTCYCLE to find valid folderId values.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
summaryYesTest cycle name/summary
descriptionNo
priorityNo
statusNo
assigneeNoAssignee Jira account ID
folderIdNoTarget folder ID
plannedStartDateNoISO 8601 planned start date
plannedEndDateNoISO 8601 planned end date
customFieldsNo

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Mentions return value (id and key), which is helpful given no output schema. Does not disclose side effects, permissions, or error conditions. Without annotations, basic behavioral disclosure is present but incomplete.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences pack purpose, return, and a usage tip without extraneous text. Efficient and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema or annotations make it essential to cover behavior and parameters. Covers return and a key parameter but lacks detail on required fields beyond schema and potential errors. Adequate for a straightforward creation tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 60% schema coverage, description adds value for folderId (explains prerequisite) and hints at return fields, but other parameters (priority, status, assignee) remain undescribed beyond the schema. Adequate but not enriching.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the verb 'Create' and resource 'new test cycle in QMetry', with return details. Distinguishes from sibling tools like update, delete, and get.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit prerequisite guidance for folderId by referencing list_folders. Does not explicitly contrast with alternatives like create_test_case or create_test_plan, but sufficiently narrows usage context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_planA

Create a new test plan in QMetry. Returns the created plan with its internal id and key (e.g. FS-TP-44). Use list_folders with folderType=TESTPLAN to find valid folderId values.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
summaryYesTest plan name/summary
descriptionNo
priorityNo
statusNo
assigneeNoAssignee Jira account ID
folderIdNoTarget folder ID
customFieldsNo

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description bears full burden. It discloses that the created plan returns internal id and key, but lacks details on idempotency, permissions, cascading effects, or error behavior. Adequate for a simple CRUD operation but could be richer.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no redundancy, front-loaded with purpose. Efficient and to the point.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 8 parameters, no output schema, and no annotations, the description is incomplete. It covers only purpose and a single parameter hint. Missing error conditions, project context, and parameter semantics for most fields.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 50% description coverage. Description only elaborates on folderId via list_folders hint. Does not explain parameters like description, priority, status, or custom fields, which are partially undocumented in schema. Insufficient compensation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the verb 'create', resource 'test plan', and system 'QMetry'. Returns specific identifiers (internal id and key). Distinguishes from sibling creation tools like create_test_case or create_test_cycle.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides explicit guidance on using list_folders to find valid folderId values. Does not explicitly state when not to use, but context is implicit for a creation tool. Could be improved by mentioning prerequisites like project existence.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

create_test_stepsB

Add one or more test steps to a specific version of a test case. Each step has stepDetails (required), expectedResult, and testData. Returns the created step objects with their IDs.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest case ID
versionNoYesTest case version number
stepsYesSteps to create

TDQS

B3.4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full burden. It discloses that steps are created and returns objects with IDs, but lacks details on side effects, permissions, or behavior if the version does not exist.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and key details. No extraneous information, though it could be slightly more structured for readability.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a creation tool with no output schema, the description covers purpose and return value. Missing context on error cases, duplication, or whether steps are appended or replaced.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the description adds minimal value beyond the schema. It reiterates that stepDetails is required and lists step fields, which are already in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the tool adds test steps to a specific version of a test case, using the verb 'add' and specifying the resource. It distinguishes from sibling tools like update_test_steps by focusing on creation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance on when to use this tool versus alternatives such as update_test_steps for modifying steps. Users are left to infer usage from the purpose alone.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_test_caseA

Permanently delete a specific version of a test case. If it is the only version, the test case is removed entirely. Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest case ID
versionNoYesVersion number to delete

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses key behavioral traits: 'permanently delete' indicates destructive action, clarifies version-specific behavior, and mentions the return code 204. With no annotations provided, it carries the full burden and does so adequately, though it could add more details like authorization requirements.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no unnecessary words. Critical information is front-loaded: the action, the target, the conditional behavior, and the return value. Every sentence earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (no output schema, two parameters), the description covers the return value (204) and the effect on test case versions. It is sufficiently complete for its complexity, though it could benefit from mentioning that the id can be either a string or number (but schema already covers that).

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description does not add extra meaning beyond the schema's property descriptions. For a simple two-parameter tool, this meets the baseline but does not elevate it.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Permanently delete a specific version of a test case' with the specific verb 'delete' and resource 'test case version'. It also explains the behavior when it's the only version, distinguishing it from sibling tools like delete_test_cycle or delete_test_plan.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives. Does not mention prerequisites, such as how to obtain the version number, or indicate that deletion is irreversible and should be used with caution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_test_cycleA

Permanently delete a test cycle and all its execution records. This is irreversible. Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest cycle ID

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description correctly discloses the irreversible nature and the HTTP 204 return code. It adds value beyond the schema by explaining the permanent effect. Minor omission: no mention of behavior on invalid IDs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with action and consequence. Every word serves a purpose, no fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple delete tool with one parameter and no output schema, the description sufficiently covers the action, effects, and return status. It is complete enough given the low complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema descriptions cover the single parameter 'id' with 100% coverage. The description adds no additional parameter-specific details beyond what the schema already provides, so baseline score applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses specific verb 'delete' and states the resource 'test cycle' along with 'all its execution records'. It clearly distinguishes from sibling tools like get_test_cycle or update_test_cycle by emphasizing permanence.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not provide explicit guidance on when to use this tool versus alternatives, nor does it mention prerequisites or cautions. However, the purpose is clear and sibling context implies it's for deletion.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_test_planA

Permanently delete a test plan. This does not delete the linked test cycles. Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest plan ID

TDQS

A3.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Discloses that deletion is permanent and returns 204, and that linked test cycles are unaffected. However, with no annotations, it lacks details on permissions, reversibility confirmation, or failure conditions.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences front-load the core action and key constraint. Every sentence adds value without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple one-parameter tool and no output schema, the description adequately covers the behavior (permanence, side effects, success response). No additional details are necessary.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the 'id' parameter. The description adds no extra semantic meaning beyond the schema's 'Test plan ID'.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states the action (permanently delete) and resource (test plan). It distinguishes from siblings by noting linked test cycles are not deleted, which is a key differentiator from delete_test_cycle.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool versus alternatives like unlinking cycles first. No prerequisites or conditions for use are mentioned.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_linked_test_cyclesA

List all test cycles currently linked to a test plan. Returns paginated list with id, key, status, priority per cycle. Use the plan's internal id (not key).

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest plan ID
startAtNoPage offset (default 0)
maxResultsNoItems per page (max 100, default 50)
sortNoSort e.g. "id:asc" or "updated:desc"
fieldsNoComma-separated fields to return

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the return fields and pagination behavior but does not mention side effects, authentication needs, or rate limits. For a read-only listing tool, the transparency is adequate but not exceptional.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with purpose and return fields. Every word earns its place. No redundancy or fluff.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity (5 params, no output schema, no annotations), the description covers the main purpose, return fields, and key parameter guidance. It lacks details on sorting and filtering parameters, but for a listing tool, it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with descriptions for all parameters. The description adds value by clarifying the 'id' parameter ('internal id, not key') and noting pagination (startAt, maxResults). This guidance goes beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('list all test cycles'), the resource ('linked to a test plan'), and the return fields. It distinguishes from siblings like 'get_test_cycle' (single) and 'search_test_cycles' (not necessarily linked) by specifying 'linked to a test plan'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides context for when to use this tool (when you need linked cycles) and gives a specific usage instruction ('Use the plan's internal id (not key)'). It does not explicitly state when not to use it or list alternative tools, but the purpose clarity implies the scope.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_caseA

Get a test case by its internal ID or key (e.g. FS-TC-31950). Returns an array of versions, each with versionNo, isLatestVersion, aiGenerated flag, and any test steps.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest case ID or key (e.g. QTP-TC-1)

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden for behavioral disclosure. It describes the return format (array of versions with fields) but omits aspects like permissions, rate limits, or error handling (e.g., what happens if ID not found).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences with no superfluous words: first states purpose, second lists returned fields. Very concise and front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description partially compensates by listing return fields (versionNo, isLatestVersion, aiGenerated, test steps). However, it does not explain edge cases (e.g., not found, multiple versions) or pagination, leaving some gaps for a complex environment with many sibling tools.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The only parameter 'id' is well-described in the input schema (100% coverage). The description adds a slight bit of context (example key format) but does not significantly enhance understanding beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the action ('Get a test case') and the resource ('by its internal ID or key'). It includes an example key format (FS-TC-31950) and explicitly distinguishes from sibling tools like search_test_cases by indicating a direct lookup.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool (when you have an ID or key) but does not explicitly mention when not to use it or suggest alternatives. Siblings like search_test_cases are not referenced for contrast.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_cycleA

Get a test cycle by its key (e.g. FS-TR-747) or internal id. Returns id, key, status, priority, projectId, archived flag. The internal 'id' returned here is required by get_test_cycle_executions and update_test_execution.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest cycle ID or key

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must convey safety. It does not explicitly state that the tool is read-only or has no side effects. However, the name and returned fields imply no mutation. Additional context on permissions or rate limits is missing.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences: first states purpose and return fields, second explains the downstream use of the id. No redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple get tool with one parameter and no output schema, the description covers purpose, input format, and return fields. It could be more complete by contrasting with search_test_cycles, but it is adequate for the complexity.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3. The description adds value by providing an example key format (e.g., FS-TR-747) and clarifying that the returned internal id is used by other tools. This aids correct usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool retrieves a test cycle by key or internal id and lists returned fields. It distinguishes from sibling tools like search_test_cycles (search) and get_test_cycle_executions (different endpoint).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use this tool by noting that the returned internal id is required by get_test_cycle_executions and update_test_execution. However, it does not explicitly compare to search_test_cycles or state when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_cycle_executionsA

List all test case executions linked to a test cycle. Requires the internal cycle id (from get_test_cycle, not the key). Returns testCycleTestCaseMapId (needed for bulk_update), testCaseExecutionId (needed for update_test_execution), key, status, and priority per test case.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest cycle ID (internal ID from search_test_cycles)
startAtNoPage offset (default 0)
maxResultsNoItems per page (max 100, default 50)
sortNoSort e.g. "id:asc" or "updated:desc"
fieldsNoComma-separated fields to return

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations; description adds context by specifying required internal ID and returned fields. However, does not disclose pagination limits, rate limits, or read-only nature beyond what schema implies. Adequate but not exhaustive.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences with no redundancy. First sentence states purpose, second provides prerequisites and output. Ideal front-loading.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema, but description lists returned fields. Covers purpose, prerequisite, and output. Lacks mention of pagination behavior or error cases, but sufficient for typical use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline 3. Description adds significant value for the 'id' parameter by clarifying it must be internal cycle ID from get_test_cycle. Other params are adequately described in schema. Overall adds meaning beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states 'List all test case executions linked to a test cycle' with specific verb and resource, and distinguishes from sibling tools like get_test_cycle or search_test_cycles.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly requires internal cycle id from get_test_cycle, providing clear prerequisite. Mentions returned IDs useful for bulk_update and update_test_execution, guiding subsequent tool use. No explicit when-not or alternatives, but guidance is strong.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_test_planA

Get a test plan by its key (e.g. FS-TP-43) or internal id. Returns id, key, projectId, archived flag. Use the internal 'id' when calling link/unlink/get_linked_test_cycles.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest plan ID or key

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description bears the burden. It lists returned fields but does not state read-only nature, error behavior, or idempotency. Adequate but minimal.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, no redundancy. Every word serves a purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple retrieval tool with one parameter, the description is nearly complete. Could mention what happens if id not found, but overall sufficient given sibling context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Adds value beyond schema by providing example key format (FS-TP-43) and advising to prefer internal id for subsequent calls. Schema coverage is 100%, but description enriches understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Clearly states the function: getting a test plan by key or internal ID. Distinguishes from siblings like get_test_case and get_test_cycle by specifying the target resource and return fields.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides a concrete usage hint: 'Use the internal id when calling link/unlink/get_linked_test_cycles.' However, does not explicitly contrast with search_test_plans for when to use each.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_foldersA

List folders of a given type in a project as a nested tree with id, name, parentId, and children. Provide folderId to return only that subtree instead of the full project tree (recommended for large projects).

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
folderTypeYesFolder type to list
folderIdNoReturn only this folder and its children (subtree). Omit to get the full project tree.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description explains that the tool returns a nested tree and supports optional subtree filtering. It is clear that this is a read operation, but does not cover permissions, rate limits, or edge cases.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core function, and no unnecessary words. Every sentence adds meaningful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the return format (nested tree with fields) and the optional subtree filter. Given no output schema, it provides adequate context for a list tool, though could mention empty results.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds value by explaining the nested tree structure and the recommendation to use folderId for large projects, clarifying the purpose beyond schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool lists folders of a given type as a nested tree, specifying the fields (id, name, parentId, children). It distinguishes from siblings like create_folder (creation) and search_test_cases (search).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description advises providing folderId for a subtree instead of the full tree, recommended for large projects. It does not explicitly state when not to use or list alternatives, but the guidance is useful.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

run_automation_rulesA

Trigger an automation rule to run against a specific test cycle. testCycleId is the internal id string (from get_test_cycle). Returns a background task object with taskId and progressUrl to poll for completion.

ParametersJSON Schema
NameRequiredDescriptionDefault
automationRuleKeyYesAutomation rule key to run
projectIdYesJira project numeric ID (e.g. 10011)
testCycleIdYesInternal test cycle ID (from search_test_cycles)

TDQS

A3.8/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must cover behavioral traits. It discloses the async nature by describing the return object (taskId, progressUrl), but does not mention side effects, permissions, or failure modes. This provides moderate transparency.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description consists of two concise sentences: one for purpose, one for parameter clarification and return value. No redundancy, perfectly front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite lacking output schema, the description explains the return format (background task with taskId and progressUrl) and references the source of testCycleId. It assumes some domain knowledge but covers essential usage points for a 3-required-param tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so baseline is 3. The description adds marginal value by specifying testCycleId as 'internal id string (from get_test_cycle),' but schema already says 'Internal test cycle ID (from search_test_cycles).' No meaningful addition for other parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'Trigger an automation rule to run against a specific test cycle,' specifying the verb (trigger), resource (automation rule), and target (test cycle). It distinguishes from sibling tools like link_automation_rule by indicating execution rather than association.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description notes that testCycleId is the internal id from get_test_cycle, implying a prerequisite, and mentions the async return result. However, it does not explicitly compare with alternatives like link_automation_rule or state when not to use this tool, leaving usage context incomplete.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_test_casesA

Search test cases in a project with optional filters. Returns total count and paginated data with id, key, version info, archived flag. Use status/priority name strings (e.g. 'To Do', 'High') not IDs. projectId must be numeric (10011). Set recursive=true with a folderId to count across all subfolders (returns total only, no data).

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
labelsNoFilter by labels
componentsNoFilter by components
recursiveNoIf true and folderId is set, counts test cases across all subfolders recursively. Returns { total, folderCount } only.
folderIdNoFilter by folder ID
statusNoFilter by status values
priorityNoFilter by priority values
assigneeNoFilter by assignee Jira account IDs
queryNoFree-text search query
startAtNoPage offset (default 0)
maxResultsNoItems per page (max 100, default 50)
sortNoSort e.g. "id:asc" or "updated:desc"
fieldsNoComma-separated fields to return

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, but description discloses key behaviors: returns total count and paginated data with specific fields, recursive mode returns only total/folderCount, projectId must be numeric. Adequate for a search tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three concise sentences, each providing essential information without redundancy. Front-loaded with purpose, then critical usage details.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers return structure (total, paginated data with fields) and behavior (recursive mode). No output schema, but description sufficiently sets expectations. Could mention default pagination, but schema covers that.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema has 100% coverage, but description adds value by clarifying that status/priority are string names, projectId is numeric, and recursive mode behavior. This goes beyond the schema descriptions.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool searches test cases in a project with optional filters and returns paginated data. It distinguishes from sibling tools (e.g., search_test_cycles, search_test_plans) by focusing on test cases.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides specific guidance on using status/priority name strings instead of IDs, and explains the recursive mode behavior. Lacks explicit alternative tool mentions, but the context is sufficient for correct invocation.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_test_cyclesA

Search test cycles in a project. Returns total count and paginated list with id, key, status, priority, archived. The 'id' field in results is the internal ID needed for execution tools.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
folderIdNoFilter by folder ID
statusNoFilter by status values
priorityNoFilter by priority values
assigneeNoFilter by assignee Jira account IDs
queryNoFree-text search query
startAtNoPage offset (default 0)
maxResultsNoItems per page (max 100, default 50)
sortNoSort e.g. "id:asc" or "updated:desc"
fieldsNoComma-separated fields to return

TDQS

A4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that the tool returns a paginated list with total count and notes that the 'id' field is the internal ID needed for execution tools. This adds value beyond the schema, though it could mention read-only nature more explicitly.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the primary action and return information, the second adds a critical detail about the id field. Every sentence provides value with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of a search tool with 10 parameters and no output schema, the description covers the core functionality and returned fields. It lacks a full list of possible returned fields and does not explain the 'fields' parameter's effect, which is partially compensated by schema descriptions.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so each parameter is already documented. The tool description does not add additional meaning to individual parameters beyond the schema descriptions. It mentions returned fields but does not elaborate on parameter usage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it searches test cycles in a project and lists specific returned fields (id, key, status, priority, archived). It distinguishes itself from siblings by focusing on search and mentioning the internal ID needed for execution tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly state when to use this tool versus alternatives (e.g., get_test_cycle or create_test_cycle). Usage is implied by the verb 'search' and the context of the tool name, but no when-not or alternative guidance is provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_test_plansB

Search test plans in a project. Returns total count and paginated list with id, key, projectId, archived. The 'id' field is the internal ID needed for linking cycles.

ParametersJSON Schema
NameRequiredDescriptionDefault
projectIdYesJira project numeric ID (e.g. 10011)
folderIdNoFilter by folder ID
statusNoFilter by status values
priorityNoFilter by priority values
assigneeNoFilter by assignee Jira account IDs
queryNoFree-text search query
startAtNoPage offset (default 0)
maxResultsNoItems per page (max 100, default 50)
sortNoSort e.g. "id:asc" or "updated:desc"
fieldsNoComma-separated fields to return

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must disclose behavioral traits. It mentions return fields but does not state read-only nature, authentication requirements, rate limits, or destructive potential.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two well-structured sentences with front-loaded purpose. No fluff, but could be more structured with bullet points.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 10 parameters, no annotations, and no output schema, the description is inadequate. It does not explain pagination behavior, search semantics, or output structure beyond a few fields.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Input schema has 100% parameter descriptions, so baseline is 3. Description does not add additional meaning beyond schema; it only mentions output fields.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly states it searches test plans in a project, which is a specific verb+resource. It distinguishes from sibling search tools by resource type (test plans vs test cases/cycles).

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides some usage context (returns paginated list, id needed for linking cycles) but no explicit guidance on when to use vs alternatives or when not to use.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_caseB

Update fields on a specific version of a test case. Requires both the test case id and versionNo (usually 1 for latest). Priority and status take integer IDs. Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest case ID
versionNoYesVersion number to update
summaryNo
preconditionNo
priorityNoPriority integer ID
statusNoStatus integer ID
assigneeNoAssignee Jira account ID
labelsNo
componentsNo
fixVersionsNo
customFieldsNo

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses that it updates fields, requires two parameters, and returns 204 on success, but it omits critical behavioral details such as whether it performs a partial update or full replacement, what happens if the version doesn't exist, authorization requirements, or error responses. With no annotations, the description carries the full burden, and it is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise with three sentences, front-loading the purpose and key requirements. It efficiently conveys core information without wordiness, earning a high score for conciseness, though a small amount of additional detail could be justified.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the complexity of 11 parameters, no output schema, and no annotations, the description provides the minimum viable information: required parameters, return code, and an example of integer IDs. However, it leaves significant gaps in understanding behavior for optional fields, error handling, and distinction from related tools, making it adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description only adds meaning for three parameters (id, versionNo, priority/status) beyond the input schema, which already provides descriptions for many parameters. With schema coverage at 45%, the description does not compensate for the missing parameter information, leaving many fields like summary, precondition, customFields unaddressed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it updates fields on a specific version of a test case, using the verb 'update' and resource 'test case version'. It distinguishes itself from sibling tools like update_test_cycle or update_test_execution by focusing on test case version and mentioning required parameters id and versionNo.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description says requires both test case id and versionNo, and notes versionNo is usually 1 for latest, giving implied usage context. However, it lacks explicit guidance on when not to use this tool (e.g., for creating or deleting a test case) or alternatives like bulk_update_test_executions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_cycleA

Update a test cycle's metadata (summary, description, priority, status, dates, custom fields). Pass the internal id or key. Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest cycle ID
summaryNo
descriptionNo
priorityNo
statusNo
assigneeNo
plannedStartDateNoISO 8601 date
plannedEndDateNoISO 8601 date
customFieldsNo

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are present, so the description bears full responsibility. It discloses the HTTP return status (204 on success), but lacks information on side effects, required permissions, error conditions, or whether updates are reversible. For a mutation tool, this is insufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two succinct sentences: first outlines the tool's purpose, second provides usage hint and response. No redundant information; every word adds value.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 9 parameters, no output schema, and no annotations, the description covers the essential purpose, identifier passing, and response status. However, it omits details like validation rules, custom field structure guidance, and error handling, which are needed for full completeness.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is low (33%), so the description must compensate. It lists the updatable fields, which adds context beyond the schema names, but it does not specify value formats or constraints for fields like priority or status. The list is helpful but incomplete.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Description clearly specifies the action ('Update'), the resource ('test cycle'), and enumerates the metadata fields that can be updated (summary, description, priority, status, dates, custom fields). This distinguishes it from sibling tools like update_test_case or update_test_plan.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides minimal guidance on how to use: 'Pass the internal id or key.' However, it does not discuss when to use this tool versus alternatives (e.g., when to update vs create a test cycle) or any prerequisites or restrictions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_executionA

Update a single test case execution result inside a test cycle. Use testCaseExecutionId from get_test_cycle_executions. executionResultId: 239443=Not Executed, 239444=Pass, 239441=Fail, 239442=Work In Progress, 239440=Blocked. Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycleIdYesTest cycle ID
testCaseExecutionIdYesTest case execution ID
executionResultIdNoExecution result/status ID
environmentIdNoEnvironment ID
buildIdNoBuild ID
commentNoExecution comment
actualTimeNoActual time spent in milliseconds

TDQS

A4.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description shoulders the burden. It states the return code (204) and implies mutation, but does not disclose permissions, destructive potential, or behavior for omitted fields. Adequate but not thorough.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences are tight, front-loaded with the main action, and every sentence adds essential information. No redundant or vague phrases.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Covers key aspects: ID source, status mapping, return code. No output schema but return is stated. Could mention error handling or partial update behavior, but for a straightforward update tool, it is sufficiently complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 100% of parameters with descriptions. The description adds value by mapping executionResultId to status names and specifying the source for testCaseExecutionId. Enhances understanding beyond schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly specifies the action 'update' and the resource 'single test case execution result inside a test cycle'. It distinguishes from sibling 'bulk_update_test_executions' by emphasizing 'single'. The purpose is evident and unambiguous.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides guidance on sourcing testCaseExecutionId from get_test_cycle_executions and gives explicit mapping for executionResultId values. Lacks explicit when-not-to-use but implicitly contrasts with bulk update. Clear context for using the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_planC

Update a test plan's priority or custom fields. Returns 204 on success.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest plan ID
priorityNo
customFieldsNo

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses the return status (204 on success) but omits other behavioral traits such as whether partial updates are supported, validation rules, or error handling when the test plan is not found. Since no annotations are provided, the description should cover these aspects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence with no redundant information. It efficiently states the action and the return code, making it easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has 3 parameters, no output schema, and no annotations, the description is too brief. It fails to explain the format of customFields, error scenarios, or whether the update is partial, leaving the agent with insufficient context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The description merely repeats the parameter names (priority, custom fields) without adding meaning beyond what the input schema shows. With only 33% schema description coverage, the description should clarify parameter values or constraints, but it does not.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it updates a test plan's priority or custom fields, differentiating it from sibling tools like create or delete. However, it does not specify whether custom fields are replaced or merged, leaving some ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives like create_test_plan or update_test_case. There is no mention of prerequisites (e.g., the test plan must exist) or conditions under which the update should be performed.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_step_executionA

Update a single step-level execution result within a test case execution. testStepExecutionId comes from get_test_cycle_executions step data. Returns 200 with updated step data.

ParametersJSON Schema
NameRequiredDescriptionDefault
cycleIdYesTest cycle ID
testStepExecutionIdYesTest step execution ID
executionResultIdNoExecution result ID
actualResultNoActual result text
commentNo

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations provided, so description must cover behavior. It mentions the return status 200 with updated data, but does not disclose error conditions, prerequisites (e.g., existence of cycleId/testStepExecutionId), or mutation consequences.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three sentences, front-loaded with the action, clear and efficient. No unnecessary words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema and omitted details like error handling, the description is fairly complete with the return status and parameter source. However, cycleId is required but its context is not explained.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema covers 80% of parameters with descriptions. The description adds value by specifying the source of testStepExecutionId (from get_test_cycle_executions), which aids correct usage. Other parameters lack additional context.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states it updates a single step-level execution result within a test case execution, and specifies the source of testStepExecutionId from get_test_cycle_executions, distinguishing it from sibling update_test_execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for updating step-level results but does not explicitly exclude or compare with alternatives like update_test_execution. No when-to-use or when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_test_stepsA

Update existing test steps on a test case version. Each step must include its step id (from create_test_steps or get_test_case response). Returns the updated step objects.

ParametersJSON Schema
NameRequiredDescriptionDefault
idYesTest case ID
versionNoYesTest case version number
stepsYesSteps to update

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Given no annotations, the description discloses that this tool modifies existing steps and returns updated objects. It doesn't detail side effects or partial update behavior, but the core update nature is clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences front-load purpose and a key prerequisite, with no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers purpose, prerequisites, and return value, but could be improved by explicitly mentioning that partial updates are allowed or that the test case version must exist.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, but the description adds valuable context that step ids must come from prior API responses, which is not explicit in the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the verb 'Update', the resource 'existing test steps on a test case version', and distinguishes from siblings like create_test_steps and update_test_step_execution.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides one key prerequisite (steps must include step id from create or get response) but does not guide when to use this tool versus alternatives like create_test_steps or update_test_step_execution.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. 30 tool updatesv1.0.0
    • First observedbulk_update_test_executions
    • First observedclone_test_cases
    • First observedcreate_folder
    • First observedcreate_test_case
    • First observedcreate_test_cycle
    • First observedcreate_test_plan
    • First observedcreate_test_steps
    • First observeddelete_test_case
    • First observeddelete_test_cycle
    • First observeddelete_test_plan
    • First observedget_linked_test_cycles
    • First observedget_test_case
    • First observedget_test_cycle
    • First observedget_test_cycle_executions
    • First observedget_test_plan
    • First observedlink_automation_rule
    • First observedlink_test_cycles_to_plan
    • First observedlist_folders
    • First observedrun_automation_rules
    • First observedsearch_test_cases
    • First observedsearch_test_cycles
    • First observedsearch_test_plans
    • First observedunlink_automation_rule
    • First observedunlink_test_cycles_from_plan
    • First observedupdate_test_case
    • First observedupdate_test_cycle
    • First observedupdate_test_execution
    • First observedupdate_test_plan
    • First observedupdate_test_step_execution
    • First observedupdate_test_steps

TDQS

A3.7/5.0
Disambiguation5/5

Each tool targets a distinct entity or operation (folders, test cases, cycles, plans, executions, automation rules, linking). Despite high tool count, descriptions make purposes clear with no overlapping functionality.

Naming Consistency5/5

All tools use a consistent verb_noun pattern in snake_case (e.g., create_folder, update_test_execution, search_test_plans). Variations between get/list/search are semantically justified by the operation type.

Tool Count3/5

30 tools is on the high side, but the domain (test management) requires many entities and operations. While borderline heavy, most tools serve essential functions without redundancy.

Completeness4/5

Covers CRUD for core entities (test cases, cycles, plans, folders), steps, executions (single/bulk), linking, and automation. Missing delete for folders and some retrieval details, but core workflows are complete.

Maintenance

ActivityInactive
ResponsivenessNo issues

Resources

Unclaimed servers have limited discoverability.

Looking for Admin?

If you are the server author, to access and configure the admin panel.

Related MCP Connectors

Related MCP Servers

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/salehrifai42/qmetrymcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server