Skip to main content
Glama

Testiny Test Management

Set the result for multiple test cases in a test run

setTestResultBatch
DestructiveIdempotent

Sets the result for multiple test cases in a test run and adds the test cases to the test run if they were not part of the test run yet

Input Schema

NameRequiredDescriptionDefault
projectYesProject ID (number or string), project key or part of the project name
resultsYes
testRunIdYesThe id of the test run

Output Schema

NameRequiredDescriptionDefault
statusYes

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • changedInput schema / properties / results / items / properties / testResult / description
      Previous value: -"The result of the test case. Builtin values include 'passed', 'failed', 'blocked', 'skipped' and 'notrun', other values can be allowed if configured. Use 'notrun' if you only want to add the test case to the run without assigning a result"New value: +"The result of the test case. The builtin values 'passed', 'failed', 'blocked', 'skipped' and 'notrun' are case-insensitive, so 'passed' and 'PASSED' are the same. A project can also define extra result statuses - listProjects reports those under the 'result_status' field, and they must be spelled exactly as reported. Use 'notrun' if you only want to add the test case to the run without assigning a result"
  2. First observed

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare the safety profile (readOnlyHint=false, destructiveHint=true, idempotentHint=true), so the bar is lower, and the description adds a genuinely useful behavioral trait beyond them: test cases absent from the run are added as a side effect. It does not, however, say that existing results are overwritten or note permission requirements, leaving the destructive aspect implicit.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single sentence that front-loads the primary action and appends the conditional side effect. It is efficient with no filler, though it is a fairly long compound sentence rather than crisply split.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need no explanation, and annotations cover the safety profile while the schema covers parameters. The remaining gap is routing guidance against setTestResult, which is the one thing an agent still has to infer.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 67% and the schema already documents all three parameters in detail, including accepted result values and the 'notrun' semantics. The description adds no additional parameter meaning beyond what the schema provides, so the baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Sets the result) and resource (multiple test cases in a test run), and the 'multiple' scope distinguishes it from the singular sibling setTestResult without naming it explicitly. An agent can infer the batch nature, though the sibling is never named.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is only implied: the batch/multi-case framing suggests this is for bulk updates versus setTestResult. There is no explicit statement of when to prefer this over the single-result sibling, no prerequisites, and no exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources