Diagnose flaky test
diagnose-flakyRepeat failing Playwright tests with retries disabled to return per-test verdicts—consistently failing, flaky, or not reproducing—and isolate parallelism bugs from real flakiness.
Instructions
Repeat failing tests with retries disabled and return a per-test, evidence-based verdict: CONSISTENTLY FAILING (real bug), FLAKY (pass/fail counts, error-signature variance, duration spread) or NOT REPRODUCING. Runs all repetitions in one process by default. isolate: true also re-runs the failures with one worker to tell parallelism bugs (shared state) from flakiness inside the test. Defaults to the tests that failed in the most recent run.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| runs | No | How many times to run the tests (default 3) | |
| config | No | playwright.config path or 1-based index when the project has several configs | |
| headed | No | Run with a visible browser window | |
| browser | No | Playwright project name to run | |
| isolate | No | Also tell parallelism bugs from real flakiness: repeat in parallel (fully parallel, `workers` default 4), then repeat the failing tests with one worker | |
| workers | No | ||
| testFiles | No | Tests to diagnose ("file:line" supported). Defaults to the tests that failed in the most recent run. | |
| timeoutMs | No | Hard wall-clock limit per run (default 120000); one --repeat-each process gets this × runs | |
| projectRoot | No | Project directory; defaults to the server working directory | |
| separateProcesses | No | Start a fresh Playwright process per run instead of one --repeat-each process (slower; use when the flake depends on global setup or a cold start) |