analyze_test_results
Analyze JUnit XML or CSV test results to compute deterministic health metrics: pass rates, flaky and ever-failing tests, never-run tests, slowest tests, and failure clusters.
Instructions
Deterministic test-results health metrics (flaky / ever-failing /
never-run / slowest / failure clustering) from real test execution
data — no LLM anywhere in this call path; write your own narrative
from the returned numbers. Provide exactly one of junit_xml or
csv_text. junit_xml is normally one JUnit XML report string for one
run (accepts both a and a bare root); to
analyze flakiness across MULTIPLE runs in one call, pass a JSON array
of {"run_id": "...", "xml": "..."} objects
(or simply a JSON array of XML strings, auto-named run-1, run-2, ...;
repeated run_ids are kept as separate runs with a #2/#3 suffix and a
warning) instead — either as a genuine JSON array/list argument, or as a string
starting with "[" (some MCP clients stringify array arguments; both
forms are accepted). csv_text needs a test-name column (name | test_name
| test | testcase | title) and a status column (status | result |
outcome); headers are matched case-insensitively. Status synonyms are
accepted (pass/ok/success, fail/failure/ko, error/err,
skip/ignored/pending). Optional columns: classname (or suite), run_id
(or run/build; default "csv"), duration_s (or duration/time/elapsed, in
seconds; a *_ms header or "350ms" values mean milliseconds), message
(or error/details). Dropped rows and unrecognised columns are listed in
warnings. If NO test could be parsed (wrong columns, empty or
header-only CSV, HTML or non-JUnit XML, malformed XML) the call returns
{"error": "no_tests_found", "message", "warnings"} instead of an empty
0% result. reference_tests, if given, is a list of test identities
("classname::name", or just "name" when there is no classname) expected
to have run — any absent from the results are reported under never_run. A
bare "name" also matches when exactly one executed test has that name
(ambiguous bare names are not guessed; a warning explains).
Skipped tests are NOT failures: they are excluded from pass rates and from
the flaky/ever-failing verdicts and are reported separately (skipped,
skipped_executions). Flaky = pass_rate (over non-skipped executions)
strictly between flaky_min and flaky_max with at least 3 such executions;
ever-failing = zero passes across at least 3 non-skipped executions.
flaky_min/flaky_max must satisfy 0 <= flaky_min < flaky_max <= 1. Tests
with fewer executions are left out of both lists; with fewer than 3 runs a
warning says so, and tests that both passed and failed are listed under
mixed_outcome (needs more runs). An empty flaky list on 1-2 runs means not
enough data, not a stable suite. Malformed/oversized input never raises — it returns
a structured {"error": "invalid_argument" | "no_tests_found", ...}.
Returns the full ResultsAnalysis as JSON (runs, total_tests, executions,
overall_pass_rate, flaky, ever_failing, never_run, slowest,
failure_clusters, per_run, warnings, skipped_executions, skipped,
mixed_outcome).
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| csv_text | No | ||
| flaky_max | No | ||
| flaky_min | No | ||
| junit_xml | No | ||
| reference_tests | No |