Skip to main content
Glama
Sign In
JevLab MCP Server

JevLab MCP Server

by omnirogue
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

Related Servers

Alternatives to JevLab MCP Server

No user-submitted related servers found.

    Related Servers

    • benchmark-sycophancy

      itsryanarmani
      A
      license
      Not graded
      quality
      B
      maintenance
      Exposes MCP tools to list and inspect the 48 harm-ladder tasks, deterministically score a submitted decision trace, run a full evaluation against a mock or real model provider, and retrieve data provenance. This lets developers measure a model's pushback and refusal calibration as plan harm rises, directly from any MCP client.
      19 npm
      MIT

    • agent-eval-gate

      dxwss
      A
      license
      Not graded
      quality
      C
      maintenance
      Serves MCP tools to inspect agent trace JSONL files and evaluate them against golden-case constraints, returning structured metrics and gate results.
      MIT

    • evalsmith-mcp

      diegoperezg7
      A
      license
      A
      quality
      A
      maintenance
      Enables MCP clients to score retrieval quality with recall, precision, nDCG, hit rate and MRR, compare 2-3 configurations with seeded paired-bootstrap confidence intervals, audit agent tool calls against expected traces, and detect prompt regressions against golden sets. It also generates Markdown, JSON or JUnit reports with pass/fail gates, all deterministically and offline without paid API keys.
      6
      172 npm
      1
      MIT

    • mcp-ml-tools

      adeoluo
      A
      license
      Not graded
      quality
      B
      maintenance
      Enables MCP clients to count tokens and estimate costs, run BM25 search, compare model evaluations with paired bootstrap and McNemar tests, validate JSON Schema output, and scan for prompt injection and PII.
      MIT
    • judgment-pack-runtime

      judgment-pack-runtimeofficial

      Judgment-Pack
      A
      license
      Not graded
      quality
      A
      maintenance
      Serves offline document validation and evaluation operations of the Judgment Pack Specification to MCP clients over stdio, enabling agents to validate and evaluate JPS documents as tool calls. Supports conformance validation, experimental evaluation with disposition and error classes, and corpus testing.
      2
      Apache 2.0

    • jevcore-mcp

      linakuznetsov47
      A
      license
      Not graded
      quality
      A
      maintenance
      Exposes three stdio tools that answer typed questions — yes/no, choice, and score — with calibrated probabilities instead of prose, letting any MCP host query a judgment model without a chat interface. Runs fully offline on a synthetic mock by default, with named, per-feature egress disclosure and redaction only when a live provider is explicitly configured.
      66 npm
      Apache 2.0
    Deploy Server
    Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

    TDQS

    B3.4/5.0

    Scored across 4 tools

    Disambiguation4/5

    Each tool targets a distinct evaluation activity: bias auditing, rubric listing, benchmarking, and calibration. The bias audit and benchmarking tools could be loosely confused as both assess judges, but the descriptions clearly distinguish bias measurement from gold-label benchmarking.

    Naming Consistency3/5

    All tools share the jevlab_ prefix and snake_case, but the naming pattern is mixed: jevlab_list_rubrics is verb_noun, jevlab_calibrate is a bare verb, jevlab_bench is a short form, and jevlab_bias_audit is a noun phrase. This is readable but not predictably consistent.

    Tool Count5/5

    Four tools is a well-scoped set for a focused judge-evaluation toolkit. Each tool covers a distinct core capability without excess or apparent redundancy.

    Completeness4/5

    The surface covers the key lifecycle steps for judge auditing: listing rubrics, auditing bias, benchmarking, and calibrating. Minor gaps exist around custom rubric management and fetching individual rubrics, but agents can work around them.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Resources