compare_paired
Compare two complete experiment arms task by task to see fixes, breaks, net score change, per-task deltas and cost or latency ratios. Judges a candidate against its parent before you record a decision.
Instructions
Compare two complete arms task by task: fixes, breaks, net score change, per-task deltas, per-group tables, and cost and latency ratios. Use it to judge a candidate against its parent; use gate_check to record a decision with noise and budget checks. Both arms must be complete, error-free and match on tasks, split, repeats and anchors. Read-only. Returns the comparison object.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"config": "<config>",
"parent_run": "<parent_run>",
"child_run": "<child_run>",
"aggregation": "<aggregation>"
},
"name": "compare_paired"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| config | Yes | Path of the scenario YAML file on the server host, absolute or relative to the server's working directory, for example `scenario.yaml`. It selects the workspace; a server started with --config rejects any other path. | |
| child_run | Yes | Run id of the candidate arm. | |
| parent_run | Yes | Run id of the parent (incumbent) arm. | |
| aggregation | Yes | How repeats combine per task: `majority` for pass/fail outcomes (ties count as failure) or `mean` for numeric scores. |