Run an LLM evaluation: load a dataset, query models via streaming, score responses with an LLM-as-judge, and return per-question scores, aggregate summary, and pass/fail status.

Check evaluation results against quality gate thresholds. Returns pass/fail per metric and overall gate status.

List past evaluation runs in a directory. Returns metadata for each run: timestamp, dataset, models, pass/fail, and cost.

Retrieve the full details of a specific evaluation run: per-question per-model scores, responses, and judge reasoning.

Compare two evaluation runs and detect regressions. Flags metrics that worsened beyond configurable tolerance.

Run retrieval metrics (recall@k, precision@k, MRR, nDCG@k) against a labelled dataset with a configurable retrieval adapter. Returns per-query metrics, dataset-level aggregate, and p50/p95 retrieval latency.

Run the full RAG pipeline: retrieve chunks, generate answers using the retrieved chunks as context, and score with context_relevance and citation_faithfulness judges. Returns retrieval metrics, generation metrics, and judge scores per query, plus an aggregate.

Compare two retrieval evaluation result files and detect drift. Flags metrics that have regressed beyond configurable tolerance. Takes two result-set paths; does not persist history itself.

[STUB - not implemented in v0.5.0] Inject poisoned chunks into a corpus and re-run retrieval evaluation. Returns a clear not-implemented response.

Generate a markdown PR comment from evaluation results. Includes results table, regression details, and threshold status.