agent-eval-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| run_evaluation_suiteC | Run RAG/wiki agent evaluation cases and emit JSON, Markdown, and telemetry outputs. |
| compare_regressionC | Compare baseline and candidate suite JSON files. |
| decide_canaryC | Apply canary promotion policy to a suite result. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 3 tools
Each tool has a distinct purpose: running evaluations, comparing results, and applying deployment policy. There is no overlap.
All tools follow a consistent verb_noun pattern (run_evaluation_suite, compare_regression, decide_canary), making them predictable.
With 3 tools, the server covers the core evaluation workflow without being bloated. Slightly minimal but appropriate for a focused server.
The tools cover the main lifecycle: run suite, compare results, decide promotion. Minor gaps like suite management exist, but the core workflow is complete.