Skip to main content
Glama

List Test Cases

list_testcase
Read-only

Lists and filters test cases across runs using run contexts like branch, time interval, or test run ID to analyze test results and failures.

Instructions

List and filter test cases across runs. Provide at least one run context: by_testrun_id, counter, by_pages, by_branch, by_time_interval, by_environment, by_author, or by_commit. Without a run context the tool returns an empty result with a warning. KEY INSIGHT: when you use by_branch, by_time_interval, by_author, or by_commit, this tool resolves the matching runs internally — you do NOT need to call list_testruns first. Combine filters freely: e.g. by_branch="main" + by_status="failed" + by_time_interval="1d" gives you all failures on main today in one call. Batch up to 20 run IDs with by_testrun_id="id1,id2,id3". Prefer specific filters and pagination over broad result sets. For a swarm run, by_machine scopes to one machine's cases, and each case carries machine_id — that is how a failure is attributed to a specific machine. There is no spec-file filter: use search with the file name (it matches every title-path segment, which includes the spec file) and sort=duration_desc to see which tests dominate a file.

Input Schema

NameRequiredDescriptionDefault
pageNo1-indexed page number (default: 1)
sortNoCase list sort order
limitNoItems per page. Data Handler accepts 10, 25, 50, or 100.
by_tagNoComma-separated tag names
searchNoSearch test title or title path
counterNoTest run counter (alternative to by_testrun_id). Single run only, e.g. 47. Use by_testrun_id for batch lookup.
by_pagesNoTest-run page for cross-run lookup (1-indexed, newest first). MCP fetches one run-list page and uses up to 20 matching runs.
by_shardNo1-based shard index — scope results to a single shard of a sharded run
by_authorNo
by_branchNo
by_commitNo
by_statusNo
projectIdYesProject ID
by_machineNoMachine (device) id, or a comma-separated list — scope results to what ran on specific machines of a swarm run. Ids come from `machines[].machine_id` in get_run_details and are case-sensitive. An unknown id matches nothing rather than erroring.
by_artifactsNo
by_testrun_idNoSingle test run ID or comma-separated (max 20)
by_environmentNo
by_testsuite_idNoFilter by suite ID
by_test_controlsNoCases Test Controls changed, within the run context: "quarantined" = on the quarantine list (`quarantined: true`), "critical" = a critical test that passed only on retry and failed the build (`critical_flaky: true`). Applied before paging.
by_time_intervalNo
by_total_runtimeNoPer-test duration filter. Numbers are SECONDS by default; suffix with `ms` for milliseconds. Examples: ">10" (>10s), "<1000ms", ">5s".
by_attempt_numberNoExact retry count filter. 0 means initial/no-retry cases (attempt_count=1), 1 means one retry (attempt_count=2).

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed35 schema fields changedv2.2.0
    • addedInput schema / $schema
      Added value: +"http://json-schema.org/draft-07/schema#"
    • addedInput schema / additionalProperties
      Added value: +false
    • removedInput schema / properties / by_artifacts / default
      Removed value: -false
    • removedInput schema / properties / by_artifacts / description
      Removed value: -"Filter test cases that have artifacts available (screenshots, videos, traces). Set to true to list only test cases with artifacts. (ID/Counter is required while using this parameter)"
    • changedInput schema / properties / by_attempt_number / description
      Previous value: -"Exact retry count filter. 0 = initial/no-retry (attempt_count=1), 1 = one retry (attempt_count=2). (ID/Counter is required while using this parameter)"New value: +"Exact retry count filter. 0 means initial/no-retry cases (attempt_count=1), 1 means one retry (attempt_count=2)."
    • addedInput schema / properties / by_attempt_number / minimum
      Added value: +0
    • changedInput schema / properties / by_attempt_number / type
      Previous value: -"number"New value: +"integer"
    • removedInput schema / properties / by_author / description
      Removed value: -"Filter by commit author name (case-insensitive, partial match). Does not require testrun_id or counter. First lists test runs by the specified author, then returns test cases from those test runs."
    • removedInput schema / properties / by_branch / description
      Removed value: -"Filter by git branch name. Does not require testrun_id or counter. First lists test runs on the specified branch, then returns test cases from those test runs. Example: 'main', 'develop'."
    • removedInput schema / properties / by_commit / description
      Removed value: -"Filter by git commit hash (full or partial). Does not require testrun_id or counter. First lists test runs with the specified commit, then returns test cases from those test runs."
    • removedInput schema / properties / by_environment / description
      Removed value: -"Filter by environment. Does not require testrun_id or counter. First lists test runs in the specified environment, then returns test cases from those test runs. Example: 'production', 'staging', 'development'."
    • addedInput schema / properties / by_machine
      Added value: +{
      +  "description": "Machine (device) id, or a comma-separated list — scope results to what ran on specific machines of a swarm run. Ids come from `machines[].machine_id` in get_run_details and are case-sensitive. An unknown id matches nothing rather than erroring.",
      +  "type": "string"
      +}
    • changedInput schema / properties / by_pages / description
      Previous value: -"List test cases by page number. Does not require testrun_id or counter. Returns test cases from all test runs on the specified page."New value: +"Test-run page for cross-run lookup (1-indexed, newest first). MCP fetches one run-list page and uses up to 20 matching runs."
    • changedInput schema / properties / by_pages / type
      Previous value: -"number"New value: +"integer"
    • changedInput schema / properties / by_shard / description
      Previous value: -"1-based shard index — scope results to a single shard of a sharded run."New value: +"1-based shard index — scope results to a single shard of a sharded run"
    • addedInput schema / properties / by_shard / exclusiveMinimum
      Added value: +0
    • changedInput schema / properties / by_shard / type
      Previous value: -"number"New value: +"integer"
    • removedInput schema / properties / by_status / description
      Removed value: -"Filter by status: 'passed', 'failed', 'flaky', 'skipped', 'interrupted', 'incomplete', or 'running'. (ID/Counter is required while using this parameter)"
    • changedInput schema / properties / by_tag / description
      Previous value: -"Filter by tag(s). Single tag or comma-separated. Example: 'smoke' or 'smoke,regression'. (ID/Counter is required while using this parameter)"New value: +"Comma-separated tag names"
    • addedInput schema / properties / by_test_controls
      Added value: +{
      +  "description": "Cases Test Controls changed, within the run context: \"quarantined\" = on the quarantine list (`quarantined: true`), \"critical\" = a critical test that passed only on retry and failed the build (`critical_flaky: true`). Applied before paging.",
      +  "enum": [
      +    "quarantined",
      +    "critical"
      +  ],
      +  "type": "string"
      +}
    • changedInput schema / properties / by_testrun_id / description
      Previous value: -"Test run ID(s). Single ID or comma-separated for multiple runs (max 20). Example: 'test_run_123' or 'run1,run2,run3'. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages)."New value: +"Single test run ID or comma-separated (max 20)"
    • changedInput schema / properties / by_testsuite_id / description
      Previous value: -"Filter by suite ID."New value: +"Filter by suite ID"
    • removedInput schema / properties / by_time_interval / description
      Removed value: -"Filter by time interval. Does not require testrun_id or counter. First lists test runs in the specified time period, then returns test cases from those test runs. Supports: '1d' (last day), '3d' (last 3 days), 'weekly' (last 7 days), 'monthly' (last 30 days), or '2024-01-01,2024-01-31' (date range)."
    • changedInput schema / properties / by_total_runtime / description
      Previous value: -"Per-test duration filter. Numbers are SECONDS by default; suffix with `ms` for milliseconds or `s` for seconds. Examples: '>10', '<1000ms', '>5s'. (ID/Counter or a run scope is required while using this parameter)"New value: +"Per-test duration filter. Numbers are SECONDS by default; suffix with `ms` for milliseconds. Examples: \">10\" (>10s), \"<1000ms\", \">5s\"."
    • changedInput schema / properties / counter / description
      Previous value: -"Test run counter number. Alternative to by_testrun_id. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages). Example: 43."New value: +"Test run counter (alternative to by_testrun_id). Single run only, e.g. 47. Use by_testrun_id for batch lookup."
    • changedInput schema / properties / counter / type
      Previous value: -[
      -  "number",
      -  "string"
      -]New value: +[
      +  "string",
      +  "number"
      +]
    • changedInput schema / properties / limit / description
      Previous value: -"Test cases per page within the resolved run(s). Snapped to the nearest of 10, 25, 50, 100 (data-handler's allowed page sizes) — other values are adjusted, not rejected. Requires a run scope (by_testrun_id, counter, or a cross-run filter); it does not resolve runs on its own."New value: +"Items per page. Data Handler accepts 10, 25, 50, or 100."
    • changedInput schema / properties / limit / type
      Previous value: -"number"New value: +"integer"
    • removedInput schema / properties / page / default
      Removed value: -1
    • changedInput schema / properties / page / description
      Previous value: -"1-indexed page number for pagination within the resolved run(s) (default: 1). Requires a run scope (by_testrun_id, counter, or a cross-run filter). To page across runs, use by_pages."New value: +"1-indexed page number (default: 1)"
    • addedInput schema / properties / page / exclusiveMinimum
      Added value: +0
    • changedInput schema / properties / page / type
      Previous value: -"number"New value: +"integer"
    • changedInput schema / properties / projectId / description
      Previous value: -"Project ID (Required). The TestDino project identifier."New value: +"Project ID"
    • changedInput schema / properties / search / description
      Previous value: -"Search test title or title path."New value: +"Search test title or title path"
    • changedInput schema / properties / sort / description
      Previous value: -"Case list sort order."New value: +"Case list sort order"
  2. Changed19 schema fields changedv2.0.2
    • changedInput schema / properties / by_attempt_number / description
      Previous value: -"Filter by attempt number. Example: 1 for first attempt, 2 for second attempt. (ID/Counter is required while using this parameter)"New value: +"Exact retry count filter. 0 = initial/no-retry (attempt_count=1), 1 = one retry (attempt_count=2). (ID/Counter is required while using this parameter)"
    • removedInput schema / properties / by_browser_name
      Removed value: -{
      -  "description": "Filter by browser name. Example: 'chromium', 'firefox', 'webkit'. (ID/Counter is required while using this parameter)",
      -  "type": "string"
      -}
    • removedInput schema / properties / by_error_category
      Removed value: -{
      -  "description": "Filter by error category. Example: 'timeout_issues', 'element_not_found', 'assertion_failures', 'network_issues'. (ID/Counter is required while using this parameter)",
      -  "type": "string"
      -}
    • removedInput schema / properties / by_error_message
      Removed value: -{
      -  "description": "Filter by error message (partial match, case-insensitive). Example: 'Test timeout of 60000ms exceeded'. (ID/Counter is required while using this parameter)",
      -  "type": "string"
      -}
    • addedInput schema / properties / by_shard
      Added value: +{
      +  "description": "1-based shard index — scope results to a single shard of a sharded run.",
      +  "type": "number"
      +}
    • removedInput schema / properties / by_spec_file_name
      Removed value: -{
      -  "description": "Filter by spec file name. Example: 'login.spec.js' or 'user-profile.spec.ts'. (ID/Counter is required while using this parameter)",
      -  "type": "string"
      -}
    • changedInput schema / properties / by_status / description
      Previous value: -"Filter by status: 'passed', 'failed', 'skipped', or 'flaky'.(ID/Counter is required while using this parameter)"New value: +"Filter by status: 'passed', 'failed', 'flaky', 'skipped', 'interrupted', 'incomplete', or 'running'. (ID/Counter is required while using this parameter)"
    • changedInput schema / properties / by_status / enum
      Previous value: -[
      -  "passed",
      -  "failed",
      -  "skipped",
      -  "flaky"
      -]New value: +[
      +  "passed",
      +  "failed",
      +  "flaky",
      +  "skipped",
      +  "interrupted",
      +  "incomplete",
      +  "running"
      +]
    • changedInput schema / properties / by_testrun_id / description
      Previous value: -"Test run ID(s). Single ID or comma-separated for multiple runs (max 20). Example: 'test_run_123' or 'run1,run2,run3'. Not required when using test run filters (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages, page, limit, get_all)."New value: +"Test run ID(s). Single ID or comma-separated for multiple runs (max 20). Example: 'test_run_123' or 'run1,run2,run3'. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages)."
    • addedInput schema / properties / by_testsuite_id
      Added value: +{
      +  "description": "Filter by suite ID.",
      +  "type": "string"
      +}
    • changedInput schema / properties / by_total_runtime / description
      Previous value: -"Filter by total runtime. Use '<60' for less than 60 seconds, '>100' for more than 100 seconds. Example: '<60', '>100', '<30'. (ID/Counter is required while using this parameter)"New value: +"Per-test duration filter. Numbers are SECONDS by default; suffix with `ms` for milliseconds or `s` for seconds. Examples: '>10', '<1000ms', '>5s'. (ID/Counter or a run scope is required while using this parameter)"
    • changedInput schema / properties / counter / description
      Previous value: -"Test run counter number. Alternative to by_testrun_id. Not required when using test run filters (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages, page, limit, get_all). Example: 43."New value: +"Test run counter number. Alternative to by_testrun_id. Not required when using a cross-run filter (by_branch, by_commit, by_author, by_environment, by_time_interval, by_pages). Example: 43."
    • changedInput schema / properties / counter / type
      Previous value: -"number"New value: +[
      +  "number",
      +  "string"
      +]
    • removedInput schema / properties / get_all
      Removed value: -{
      -  "default": false,
      -  "description": "Get all results up to 1000 (default: false). Does not require testrun_id or counter. When used alone, first lists all test runs, then returns test cases from those test runs.",
      -  "type": "boolean"
      -}
    • removedInput schema / properties / limit / default
      Removed value: -1000
    • changedInput schema / properties / limit / description
      Previous value: -"Number of results per page (default: 1000, max: 1000). Does not require testrun_id or counter. When used alone, first lists test runs, then returns test cases from those test runs."New value: +"Test cases per page within the resolved run(s). Snapped to the nearest of 10, 25, 50, 100 (data-handler's allowed page sizes) — other values are adjusted, not rejected. Requires a run scope (by_testrun_id, counter, or a cross-run filter); it does not resolve runs on its own."
    • changedInput schema / properties / page / description
      Previous value: -"Page number for pagination (default: 1). Does not require testrun_id or counter. When used alone, first lists test runs on the specified page, then returns test cases from those test runs."New value: +"1-indexed page number for pagination within the resolved run(s) (default: 1). Requires a run scope (by_testrun_id, counter, or a cross-run filter). To page across runs, use by_pages."
    • addedInput schema / properties / search
      Added value: +{
      +  "description": "Search test title or title path.",
      +  "type": "string"
      +}
    • addedInput schema / properties / sort
      Added value: +{
      +  "description": "Case list sort order.",
      +  "enum": [
      +    "name_asc",
      +    "name_desc",
      +    "duration_asc",
      +    "duration_desc"
      +  ],
      +  "type": "string"
      +}
  3. First observed

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations only declare readOnlyHint=true; the description adds behavior the annotations cannot carry — silent empty-result-with-warning semantics, internal run resolution for branch/time/author/commit filters, the 20-ID batch cap, and non-erroring behavior for unknown machine ids. The "KEY INSIGHT" about avoiding a redundant list_testruns call is exactly the kind of operational context that changes how an agent sequences calls.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The gating rule and KEY INSIGHT are front-loaded, and the example filters make the guidance actionable. It is a dense block of prose that runs long, but each sentence carries a distinct constraint or affordance rather than restating the name.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 22-parameter read-only list tool with no output schema, the definition covers the critical gating rule, filter interaction, and pagination preference, and it notes that returned cases carry machine_id. It does not describe the shape of the returned case list or paging metadata, so an agent must infer result structure.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 68%, and the description compensates for the cross-cutting parameters: it explains that by_branch/by_time_interval/by_author/by_commit implicitly resolve runs, how by_testrun_id batches ("id1,id2,id3"), and how by_machine supports failure attribution. Several parameters (by_artifacts, by_environment, by_testsuite_id, by_shard) remain undocumented, and the prose says by_machine scopes to "one machine's cases" while the schema allows a comma-separated list — a minor tension.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource (list/filter test cases) with an explicit scope ("across runs"), which separates it from the run-scoped and manual-case siblings. It also names other tools it interacts with (list_testruns, search), so an agent can place it without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly states the precondition (at least one run-context filter), the failure mode when omitted (empty result plus warning), which alternatives to avoid calling first (list_testruns), and the fallback when a filter does not exist ("no spec-file filter: use search with the file name"). It even gives a concrete combined-filter recipe, leaving nothing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.