EvalKit MCP Server
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| ANTHROPIC_API_KEY | No | Your Anthropic API key for accessing the classifier. Optional for dry run, required for live evaluation. |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| eval_classifier_robustnessC | Process a query through the configured evaluation stages. Returns a formatted analysis with per-segment routing information, aggregate metrics, and processing results. |
| eval_decompose_queryC | Split a query into conceptual sub-components for analysis. Divides the input into constituent parts based on topic boundaries. |
| eval_obfuscation_evasionB | Analyze character-level text transformations. Applies configurable character substitutions and reports detected patterns in the input text. |
| eval_statusA | Check server configuration and connection status. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 4 tools
Each tool targets a distinct evaluation function: robustness classification, query decomposition, obfuscation analysis, and server status. No overlap in purpose.
All tools share the 'eval_' prefix, but the suffixes vary in pattern (noun_noun for classifier_robustness, verb_noun for decompose_query, noun_noun for obfuscation_evasion, and single noun for status). Mostly consistent with minor deviation.
Four tools is well-scoped for an evaluation kit, covering multiple evaluation stages without excess or deficiency.
The set covers core evaluation operations and status checking, though a tool for retrieving historical results or configuring evaluations would round it out. Minor gap, not severe.