Skip to main content
Glama

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
EVALGATE_DIRNoPath to the evalgate checkout. Defaults to ../evalgate.../evalgate

Capabilities

Features and capabilities supported by this server

CapabilityDetails
tools
{
  "listChanged": true
}

Tools

Functions exposed to the LLM to take actions

NameDescription
list_suitesA

Load and validate the eval suites in a directory or file. Returns each suite name, its cases, the assertions declared on each case, and the threshold policy that gates it. Read-only — no model calls, no cost.

explain_resultA

Read an evalgate result artifact (.evalgate/result.json) and explain why it passed or failed: which gates tripped, which cases failed, and the per-assertion explanation for each — including the claim-level grounding breakdown. Read-only.

check_driftA

Analyze .evalgate/history.jsonl for slow decline the per-PR regression gate cannot see. A case sliding a little each run passes every individual gate by construction. Reports per-series delta and slope. Read-only.

judge_statusA

Read .evalgate/calibration.json and report the judge’s measured agreement with human scores. An uncalibrated judge is a random number generator with good manners — call this before trusting any rubric or grounded score. Read-only.

run_suiteA

Run the suites at a path against a system-under-test module and return the scored result. EXECUTES the SUT module and may call model providers, so it can incur real cost. Writes nothing — the result is returned in memory, not saved as an artifact.

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription

No resources

Latest Blog Posts

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/HolisticShort/evalgate-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server