Compare Perf To Baseline
compare-perf-to-baselineCompare a run (runId) or filtered target (testchimpId plus optional dimensions) to its promoted baseline. envClass is required. Optional thresholds override max p95 regression percent and maximum fail-rate increase.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"envClass": "<envClass>"
},
"name": "compare-perf-to-baseline"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| botId | No | QA bots: your botId from your stored TestChimp binding. | |
| runId | No | ||
| dataset | No | ||
| llmMode | No | ||
| profile | No | ||
| envClass | Yes | ||
| environment | No | ||
| testchimpId | No | ||
| projectApiKey | No | QA bots: your own project API key from your stored TestChimp binding. Required on every call except get-bot-credentials, get-project-credentials and get-bot-compat. Never print it. | |
| maxFailRateIncrease | No | ||
| maxP95RegressionPercent | No |