Skip to main content
Glama
Sign In
TestDino MCP

TestDino MCP

by testdino-hq
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

Get Flake Verdict

get_flake_verdict
Read-only

Determine whether a failing test is deterministic, flaky, or inconclusive by comparing retry attempt signatures within a run; requires retries enabled.

Instructions

Say whether a failing test behaves the same way every time, by comparing its retry attempts within one run. If you are debugging a failing test, call get_debug_evidence first — it returns this plus the regression boundary and every artifact link in one call, so calling this separately afterwards repeats work already done. Returns a computed verdict — "deterministic" (every attempt failed with the same signature, so the failure repeats), "flaky" (an attempt passed on retry, so the outcome is not consistent), or "inconclusive" (too few attempts, or the attempts failed differently) — plus the per-attempt signatures behind it. The verdict describes the behaviour, not the cause. It tells you which fixes the evidence cannot support; it does not tell you where the fix goes. Decide that after reading the artifacts, the trace and the code. Needs a test that ran with retries enabled; a single attempt is always inconclusive.

Input Schema

NameRequiredDescriptionDefault
projectIdYesProject ID (e.g. project_<id>)
testrun_idNoRun scope. Omit to use the most recently started run carrying this case.
testcase_idYesPlaywright pw_test_id of the failing case

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed2 schema fields changedv2.2.0
    • addedInput schema / $schema
      Added value: +"http://json-schema.org/draft-07/schema#"
    • addedInput schema / additionalProperties
      Added value: +false
  2. Addedv2.0.3

TDQS

A4.8/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=true, and the description adds substantial context beyond that: the exact semantics of each verdict, the precondition about retries, and an explicit scope limit stating the verdict describes behaviour and not cause, and cannot tell you where the fix goes. This is exactly the disclosure that helps an agent avoid over-reading the output.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core definition and the routing instruction are front-loaded, and the verdict semantics follow in a compact parenthetical. It is somewhat long for a three-parameter tool and the closing sentences about cause-vs-behaviour are a little repetitive, but each part carries real information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries the return-value burden, and it does: it names all three verdicts, explains what each means, and mentions the per-attempt signatures returned alongside. Prerequisites and the relationship to get_debug_evidence are covered, leaving nothing an agent needs to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and testrun_id already documents its omit-to-default behaviour, so the baseline is 3. The description adds meaning by framing the whole comparison as happening 'within one run', which explains why the run-scoping parameter matters, but it does not add format or edge-case detail beyond that.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource: it decides whether a failing test behaves the same way every time by comparing retry attempts within a run. It is clearly distinguishable from siblings like get_trace_analysis and get_debug_evidence, and it even enumerates the three possible verdicts so the agent knows exactly what decision this tool yields.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes the agent: if debugging a failing test, call get_debug_evidence first because it returns this verdict plus the regression boundary and artifact links, and calling this separately afterwards repeats work. It also states a prerequisite (a test that ran with retries enabled) and the inconclusive condition for single attempts.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools