eval_compare_unpaired
Compare accuracy for two models scored on separate samples with a two-proportion z-test and confidence interval, showing whether the observed gap is statistically real.
Instructions
Is A's win over B real when each was scored on a different sample? Two-proportion z-test with a CI on the gap.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"n_a": "<n_a>",
"correct_a": "<correct_a>",
"n_b": "<n_b>",
"correct_b": "<correct_b>"
},
"name": "eval_compare_unpaired"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| n_a | Yes | ||
| n_b | Yes | ||
| alpha | No | ||
| correct_a | Yes | ||
| correct_b | Yes |