Compare Perf To Baseline
compare-perf-to-baselineCompare a run (runId) or filtered target (testchimpId plus optional dimensions) to its promoted baseline. envClass is required. Optional thresholds override max p95 regression percent and maximum fail-rate increase.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| botId | No | QA bots: your botId from your stored TestChimp binding. | |
| runId | No | ||
| dataset | No | ||
| llmMode | No | ||
| profile | No | ||
| envClass | Yes | ||
| environment | No | ||
| testchimpId | No | ||
| projectApiKey | No | QA bots: your own project API key from your stored TestChimp binding. Required on every call except get-bot-credentials, get-project-credentials and get-bot-compat. Never print it. | |
| maxFailRateIncrease | No | ||
| maxP95RegressionPercent | No |