can I ship
can_i_shipDetermine if a change is ready to ship by applying thresholds to a finished eval run and returning ship/hold with required versus actual check values.
Instructions
Does the evidence let this change through? Reads the gate on a finished eval run (the latest DONE run when eval_id is omitted) with your thresholds — or sensible defaults: the certified-switch test (noninferiority_margin 0.05) for a criterion run against stored answers, min_win_rate 0.5 for a comparison — and returns ship/hold with every check's required vs actual, where 'actual' is the interval's LOWER bound. Read-only; the same call is the CI step.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| model | No | Restrict the checks to one candidate key. | |
| eval_id | No | The run to gate. Default: the newest DONE run. | |
| min_win_rate | No | Comparison runs: every candidate's win-rate CI floor must be ≥ this. | |
| min_pass_rate | No | Criterion runs: corrected pass-rate CI floor must be ≥ this. | |
| noninferiority_margin | No | Certified switch: candidate's corrected pass-rate floor ≥ incumbent's rate − margin (needs a stored baseline and a calibrated judge). | |
| min_assertion_pass_rate | No | Exact all-assertions pass rate must be ≥ this. |