Analyze rerank NDCG
analyze_rerank_ndcgRead-only
Compare NDCG@k for bi-encoder vs cross-encoder ordering on labeled queries and report the delta, showing whether reranking improves or harms ranking quality.
Instructions
NDCG@k under bi-encoder ordering vs cross-encoder ordering, averaged across labeled queries (ground_truth_relevance IS NOT NULL). Reports the delta (cross - bi) — positive = the rerank is adding real ranking quality, negative = it's hurting. Surfaces rerank_hurts_ndcg (CRITICAL) or rerank_lifts_ndcg (GOOD evidence). Reads from mcpg_rag.rerank_events; returns zero counts when no labeled rows exist in the window.
Tool call template
This tool declares no required parameters. Add optional parameters from the input schema as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {},
"name": "analyze_rerank_ndcg"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| k | No | ||
| days | No | ||
| model | No | ||
| database | No | Optional: a database id from list_databases, to target a configured secondary (read-only) database; omit for the primary. Database names that are not configured ids are rejected. | |
| retrieval_index | No |
Output Schema
| Name | Required | Description | Default |
|---|---|---|---|
| k | Yes | ||
| delta | Yes | ||
| model | Yes | ||
| findings | Yes | ||
| window_days | Yes | ||
| retrieval_index | Yes | ||
| labeled_query_count | Yes | ||
| ndcg_at_k_under_bi_order | Yes | ||
| ndcg_at_k_under_cross_order | Yes |