rerank_evaluate
Evaluate RAG retrieval: rank passages against a query, compute Precision@k / Recall@k and a PASS/FAIL verdict from your relevance labels (INCONCLUSIVE without them). Modes: (1) give every passage the score YOUR reranker produced — deterministic, offline, the mode to gate CI on; (2) no scores, no key — a local BM25 keyword baseline, never a judgement of a neural reranker; (3) api_key for an NVIDIA account that still has reranking (NVIDIA retired the hosted endpoints on 2026-05-18).
Example tool call
This call uses example arguments supplied in the tool’s input schema.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"query": "capital of France",
"api_key": "<your-api-key>",
"passages": [
{
"text": "Paris is the capital of France.",
"relevant": true
},
{
"text": "Bananas are rich in potassium.",
"relevant": false
}
]
},
"name": "rerank_evaluate"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | ||
| top_k | No | Default 3. | |
| api_key | No | NVIDIA key, used only when no passage has a score; never stored. | |
| passages | Yes | 2 to 20 passages. | |
| threshold | No | Minimum Precision@k to PASS (default 0.5). |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| k | No | ||
| mode | No | ||
| model | No | ||
| ranked | No | ||
| verdict | No | ||
| threshold | No | ||
| latency_ms | No | ||
| recall_at_k | No | ||
| precision_at_k | No | ||
| total_passages | No | ||
| has_ground_truth | No |