groundcheck_compare
Read-only
Compare two candidate answers to a query and determine which is better based on faithfulness, completeness, and relevance, with automatic mitigation of position bias.
Instructions
Judge which of two candidate answers to query is better.
Use this to A/B two RAG configurations (prompts, retrievers, models) on
the same query. Position bias is mitigated automatically: the judge sees
(A,B) and (B,A) in separate calls, and any criterion whose verdict flips
with presentation order is reported as a tie rather than a pick.
Args:
query: the shared query both answers respond to.
answer_a: first candidate answer.
answer_b: second candidate answer.
sources: optional list of {id, text} chunks, used to judge faithfulness.
criteria: judged criteria (default ["faithfulness", "completeness", "relevance"]).
Returns a winner ("a"/"b"/"tie/uncertain"), a verdict per criterion, and a
brief rationale. Costs 2 model calls via your client's sampling -- no API
key needed if your client supports sampling.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| sources | No | ||
| answer_a | Yes | ||
| answer_b | Yes | ||
| criteria | No |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| winner | Yes | ||
| criteria | Yes | ||
| rationale | Yes | Brief overall rationale, notes disagreement if any. |