Enables building and running custom LLM benchmarks with multi-judge evaluation, supporting GUI, MCP client, and CLI usage for ranked, auditable results.
Exposes Okareo's evaluation capabilities as MCP tools, enabling AI coding assistants to manage scenarios, register models, run evaluations, and execute multi-turn simulations directly from the editor.
An MCP server that evaluates whether retrieval methods and AI outputs are grounded in long narrative manuscripts by retrieving evidence and scoring coverage deterministically, without external model APIs. It provides tools for chunking, indexing, retrieval, and evaluation.
An evaluation harness that probes MCP-based retrieval servers for calibration, relevance, coverage, citation integrity, and more, also exposing the probes as MCP tools for assistants and CI agents.
Enables LLM evaluation and observability by uploading documents, building test sets, running RAG pipelines, and automatically scoring answers for groundedness, hallucination risk, retrieval quality, latency, and cost, with tools exposed to MCP-compatible clients.