Skip to main content
Glama
Sign In
TestDino MCP

TestDino MCP

by testdino-hq
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Remote

Verify Fix

verify_fix
Read-only

Check whether a test fix held by comparing its run history after a baseline against before, returning fixed, not_fixed, changed_failure, unstable, or no_runs_since_baseline.

Instructions

Check whether a fix actually held for one test, against the run you saw when you proposed it. Splits the test's run history at that baseline and compares after against before, returning "fixed" (passing with no retries since), "not_fixed" (still failing with the same error), "changed_failure" (still failing, but a different error — a new investigation, and only when every failure since carried a comparable fingerprint), "still_failing" (still failing, but the errors cannot be compared, so neither same nor different can be claimed), "unstable" (passing only after retries, which is not fixed), "no_runs_since_baseline", or "baseline_not_found" (the run id is not one this test executed in). Call this after a new run lands. An unchanged error means the fix missed, not that the test is flaky. Identify the test by testcase_id (its pw_test_id, exact) or testcase_name; a title shared by several Playwright projects returns 409 AMBIGUOUS_IDENTITY (candidates name each project), so prefer testcase_id when you hold it. The baseline run must be one this test actually executed in — an id from another project or another test is rejected rather than answered.

Input Schema

NameRequiredDescriptionDefault
projectIdYesProject ID (e.g. project_<id>)
testcase_idNoThe test's pw_test_id — exact, never ambiguous. Wins over testcase_name.
testcase_nameNoFull test title, same identifier debug_testcase takes. Give this or testcase_id.
baseline_run_idYesThe run you saw the failure in when you proposed the fix
suite_file_pathNoSpec file path — only needed when the title is shared across files

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed5 schema fields changedv2.2.0
    • addedInput schema / $schema
      Added value: +"http://json-schema.org/draft-07/schema#"
    • addedInput schema / additionalProperties
      Added value: +false
    • addedInput schema / properties / testcase_id
      Added value: +{
+  "description": "The test's pw_test_id — exact, never ambiguous. Wins over testcase_name.",
+  "type": "string"
+}
    • changedInput schema / properties / testcase_name / description
      Previous value: -"Full test title, same identifier debug_testcase takes"New value: +"Full test title, same identifier debug_testcase takes. Give this or testcase_id."
    • changedInput schema / required
      Previous value: -[
-  "projectId",
-  "testcase_name",
-  "baseline_run_id"
-]New value: +[
+  "projectId",
+  "baseline_run_id"
+]
  2. Addedv2.0.3

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint, but the description discloses a great deal beyond that: the seven possible result states and their meanings, the 409 AMBIGUOUS_IDENTITY ambiguity case, and the fact that a baseline id from another test/project is rejected rather than answered. This is exactly the behavioral context an agent needs and cannot get from the schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core operation, then the return-value enumeration, then identity resolution and error cases. Dense and mostly waste-free, though the long comma-separated enumeration of statuses is heavy; it is justified because no output schema exists.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must carry the return contract, and it enumerates every status and edge case thoroughly. Identity resolution, baseline validation, and timing guidance are all present, so an agent has everything needed to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so the baseline is 3, and the description adds genuine meaning on top: testcase_id wins over testcase_name, the shared-title ambiguity case, and the conditional nature of suite_file_path. It stops short of documenting baseline_run_id formats or projectId, but adds real value beyond the schema text.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('check whether a fix actually held'), the exact scope ('one test'), and the comparison basis ('against the run you saw when you proposed it'). It is clearly distinguishable from diagnosis-oriented siblings like get_flake_verdict or debug_testcase by framing the operation as post-fix verification.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives an explicit trigger ('Call this after a new run lands') and a strong interpretive exclusion ('An unchanged error means the fix missed, not that the test was flaky'). It does not explicitly name an alternative tool to use instead, so it falls short of the top band.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools