is my judge trustworthy
is_my_judge_trustworthyGet a trust verdict for your evaluation judges by examining TPR/TNR intervals, catch rates, kappa, measurement history, and drift to see if they reliably catch real failures and pass clean ones.
Instructions
Can this workspace's judges be believed? For one criterion (pass criterion_id) or all of them: the trust verdict read from the TPR/TNR intervals (trustworthy / under-measured / borderline / misaligned / unmeasured), how often it catches real failures and passes clean ones with 95% intervals, κ, how many grades it was measured on and when, drift status, and the one action that changes its state. Read-only.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| criterion_id | No | One criterion's id for its full numbers; omit for every judge in the workspace. |