groundcheck_evaluate_retrieval
Read-only
Compute retrieval quality metrics such as precision, recall, MRR, and NDCG. With gold labels, metrics are instant; without, the tool grades relevance automatically.
Instructions
Score retrieval quality for retrieved chunks against query.
Two modes, chosen automatically:
- Mode A (pass `relevant_ids`): precision@k, recall@k, MRR, NDCG computed
by pure math from your gold relevance labels. Instant, no model calls.
- Mode B (omit `relevant_ids`): no gold labels available, so each chunk
is graded 0-3 for relevance via one sampling call, then the same
metrics are computed from those grades. Use this when you don't have
a labeled relevant-docs set for this query.
Args:
query: the search query the chunks were retrieved for.
retrieved: ranked list of {id, text} chunks, in retrieval order (rank matters).
relevant_ids: ids of chunks known to be relevant. Supply this whenever
you have gold labels -- Mode A is free and exact.
k_values: cutoffs to compute metrics at (default [3, 5, 10]).
Output states which mode ran. Mode A: instant. Mode B: 1 model call, no
API key needed if your client supports sampling.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| k_values | No | ||
| retrieved | Yes | ||
| relevant_ids | No |
Output Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| mrr | Yes | ||
| mode | Yes | Which mode actually ran: gold_labels or llm_graded. | |
| ndcg | Yes | NDCG@k for each requested k. | |
| metrics | Yes | Precision/recall@k for each requested k. | |
| graded_relevance | No | Per-chunk LLM grades, only present in llm_graded mode. |