pick_best_answer
Ranks candidate answers to a question using a judge model, returning the top answer, full ranking, and explanation for the selection.
Instructions
Given multiple candidate answers to a question, ask a judge model to rank them and identify the best one. Returns winner, ranking, and explanation.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"question": "<question>",
"candidate_answers": "<candidate_answers>"
},
"name": "pick_best_answer"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | The question the candidate answers are responding to. | |
| candidate_answers | Yes | List of candidate answers to rank. | |
| judge_model_alias | No | Model to act as judge. Defaults to the first model in models.yaml. | |
| rubric | No | Optional evaluation rubric (e.g. 'Prioritize accuracy over brevity'). | |
| temperature | No | Sampling temperature for the judge. Default 0.2. | |
| max_tokens | No | Max tokens for the judge response. Default 2048. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||