Provides advanced evaluation tools for assessing AI safety, alignment, and performance of LLM outputs. Enables programmatic evaluation of quality, safety metrics like toxicity and PII detection, and operational metrics including carbon footprint and cost estimation.
MCP-native auditor for LLM hallucination and grounding issues in RAG systems. Provides prioritized findings in table, JSON, or SARIF format for CI gating and AI agent integration.
Eval-integrity statistics for AI benchmark claims — multiple-testing correction, power/MDE for model gaps, judge-bias and leaderboard-rank checks. Catches a
benchmark number that won't survive a second look.
Evaluates RAG outputs on faithfulness, answer relevancy, and context precision using an LLM-as-a-Judge backend. Exposes tools for running evaluations, scoring individual samples, and checking thresholds, enabling CI gating and on-demand assessment via MCP.
An MCP server that audits LLM-as-judge evaluations, detecting judge drift across runs, measuring bias through controlled probes, and comparing judge agreement with human raters.
Enables acceptance gates for AI coding-agent runs by recording evidence, running deterministic validation, applying a quality gate, and rendering auditable outcomes.
