JevLab MCP Server
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": false
} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| jevlab_benchB | Benchmark recorded judges on synthetic gold labels; default is an offline demo. |
| jevlab_calibrateB | Fit Platt, temperature and histogram scaling; return holdout reliability and thresholds as JSON. |
| jevlab_bias_auditB | Measure position, length, format and provenance-based self-preference; default is synthetic. |
| jevlab_list_rubricsA | Return the ten public Apache-2.0 rubric templates. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 4 tools
Each tool targets a distinct evaluation activity: bias auditing, rubric listing, benchmarking, and calibration. The bias audit and benchmarking tools could be loosely confused as both assess judges, but the descriptions clearly distinguish bias measurement from gold-label benchmarking.
All tools share the jevlab_ prefix and snake_case, but the naming pattern is mixed: jevlab_list_rubrics is verb_noun, jevlab_calibrate is a bare verb, jevlab_bench is a short form, and jevlab_bias_audit is a noun phrase. This is readable but not predictably consistent.
Four tools is a well-scoped set for a focused judge-evaluation toolkit. Each tool covers a distinct core capability without excess or apparent redundancy.
The surface covers the key lifecycle steps for judge auditing: listing rubrics, auditing bias, benchmarking, and calibrating. Minor gaps exist around custom rubric management and fetching individual rubrics, but agents can work around them.