pick_best_answer
Ranks candidate answers to a question using a judge model, returning the top answer, full ranking, and explanation for the selection.
Instructions
Given multiple candidate answers to a question, ask a judge model to rank them and identify the best one. Returns winner, ranking, and explanation.
Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| question | Yes | The question the candidate answers are responding to. | |
| candidate_answers | Yes | List of candidate answers to rank. | |
| judge_model_alias | No | Model to act as judge. Defaults to the first model in models.yaml. | |
| rubric | No | Optional evaluation rubric (e.g. 'Prioritize accuracy over brevity'). | |
| temperature | No | Sampling temperature for the judge. Default 0.2. | |
| max_tokens | No | Max tokens for the judge response. Default 2048. |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||