evalgate-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| EVALGATE_DIR | No | Path to the evalgate checkout. Defaults to ../evalgate. | ../evalgate |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| list_suitesA | Load and validate the eval suites in a directory or file. Returns each suite name, its cases, the assertions declared on each case, and the threshold policy that gates it. Read-only — no model calls, no cost. |
| explain_resultA | Read an evalgate result artifact (.evalgate/result.json) and explain why it passed or failed: which gates tripped, which cases failed, and the per-assertion explanation for each — including the claim-level grounding breakdown. Read-only. |
| check_driftA | Analyze .evalgate/history.jsonl for slow decline the per-PR regression gate cannot see. A case sliding a little each run passes every individual gate by construction. Reports per-series delta and slope. Read-only. |
| judge_statusA | Read .evalgate/calibration.json and report the judge’s measured agreement with human scores. An uncalibrated judge is a random number generator with good manners — call this before trusting any rubric or grounded score. Read-only. |
| run_suiteA | Run the suites at a path against a system-under-test module and return the scored result. EXECUTES the SUT module and may call model providers, so it can incur real cost. Writes nothing — the result is returned in memory, not saved as an artifact. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/HolisticShort/evalgate-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server