Compare paired agent evaluation runs
compare_evaluation_runsCompares caller-reported baseline and candidate outcomes on matching case IDs, identifies coverage exclusions, and warns about low sample sizes.
Instructions
Compare bounded caller-reported outcomes on exact matching case IDs, with coverage exclusions and low-sample warnings. No API calls, statistical significance claims or stored results.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"baseline": "<baseline>",
"candidate": "<candidate>"
},
"name": "compare_evaluation_runs"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| baseline | Yes | ||
| candidate | Yes |