evalgate-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| EVALGATE_DIR | No | Path to the evalgate checkout. Defaults to ../evalgate. | ../evalgate |
Capabilities
Features and capabilities supported by this server
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| list_suitesA | Load and validate the eval suites in a directory or file. Returns each suite name, its cases, the assertions declared on each case, and the threshold policy that gates it. Read-only — no model calls, no cost. |
| explain_resultA | Read an evalgate result artifact (.evalgate/result.json) and explain why it passed or failed: which gates tripped, which cases failed, and the per-assertion explanation for each — including the claim-level grounding breakdown. Read-only. |
| check_driftA | Analyze .evalgate/history.jsonl for slow decline the per-PR regression gate cannot see. A case sliding a little each run passes every individual gate by construction. Reports per-series delta and slope. Read-only. |
| judge_statusA | Read .evalgate/calibration.json and report the judge’s measured agreement with human scores. An uncalibrated judge is a random number generator with good manners — call this before trusting any rubric or grounded score. Read-only. |
| run_suiteA | Run the suites at a path against a system-under-test module and return the scored result. EXECUTES the SUT module and may call model providers, so it can incur real cost. Writes nothing — the result is returned in memory, not saved as an artifact. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
Latest Blog Posts
- Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)By Om-Shree-0709 on .mcpAgent IdentityOAuth 2.1
- Your AI Chatbot Just Exposed Your CEO's Salary to an InternBy Om-Shree-0709 on .Agent IdentityMCP SecurityOAuth Delegation
- Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)By Om-Shree-0709 on .Agentic AiPrompt InjectionWebAssembly
MCP directory API
We provide all the information about MCP servers via our MCP API.
curl -X GET 'https://glama.ai/api/mcp/v1/servers/HolisticShort/evalgate-mcp'
If you have feedback or need assistance with the MCP directory API, please join our Discord server