eval_min_detectable_gap
Calculate the smallest accuracy gain over a baseline that an eval with n test cases can reliably detect, enabling you to sanity-check whether an existing eval set is adequately powered.
Instructions
Smallest accuracy improvement over baseline_acc that an eval with n test cases can reliably detect. Use this to sanity-check an existing eval set.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"n": "<n>",
"baseline_acc": "<baseline_acc>"
},
"name": "eval_min_detectable_gap"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| n | Yes | ||
| alpha | No | ||
| power | No | ||
| paired | No | ||
| discordance | No | ||
| baseline_acc | Yes |