judges_replay
Create a scoring run for the current judge over a dataset's existing outputs (wraps runs_create with prompt_id omitted and output_column supplied). This only sets up the run; call runs_generate to actually re-judge the outputs so you can compare against human verdicts.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"name": "<name>",
"metric_id": "<metric_id>",
"dataset_id": "<dataset_id>",
"judge_model": "<judge_model>"
},
"name": "judges_replay"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| name | Yes | ||
| metric_id | Yes | ||
| dataset_id | Yes | ||
| judge_model | Yes | ||
| output_column | No | Dataset column with the existing outputs to grade. Defaults to actual_output. |