evaluate_rag_end_to_end
Run a complete RAG pipeline that retrieves chunks, generates answers, and scores them on context relevance and citation faithfulness. Returns per-query and aggregate metrics.
Instructions
Run the full RAG pipeline: retrieve chunks, generate answers using the retrieved chunks as context, and score with context_relevance and citation_faithfulness judges. Returns retrieval metrics, generation metrics, and judge scores per query, plus an aggregate.
Tool call template
Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.
{
"id": 1,
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"arguments": {
"dataset_path": "<dataset_path>",
"corpus_path": "<corpus_path>",
"models": "<models>"
},
"name": "evaluate_rag_end_to_end"
}
}Input Schema
| Name | Required | Description | Default |
|---|---|---|---|
| k | No | ||
| judge | No | ||
| models | Yes | Models to evaluate. | |
| adapter | No | bm25 | |
| output_dir | No | ||
| corpus_path | Yes | ||
| dataset_path | Yes |