Skip to main content
Glama
berkayildi

mcp-llm-eval

by berkayildi
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

Related Servers

Alternatives to mcp-llm-eval

No user-submitted related servers found.

    Related Servers

    • mcp-prompt-lab

      nikolasbarwicki
      A
      license
      Not graded
      quality
      D
      maintenance
      A local MCP server for prompt evaluation, enabling users to define test cases, run prompts against multiple LLM providers, score outputs with deterministic and LLM-graded assertions, and track quality over time, all within an AI coding environment.
      MIT

    • groundcheck

      offquestxo
      A
      license
      A
      quality
      B
      maintenance
      An MCP server that lets any AI agent evaluate RAG outputs -- faithfulness scoring, hallucination detection, and retrieval quality metrics -- with zero API keys, using MCP sampling.
      6
      MIT

    • WhitePact

      Guruprasath-Annadurai
      A
      license
      A
      quality
      A
      maintenance
      An MCP server that exposes governance, trust-scoring, compliance, guardrail, cost, drift, and supply-chain scanning tools and resources to any MCP client over stdio, Streamable HTTP, or legacy HTTP+SSE. It lets agents route every tool call through a deterministic five-way decision (allow, redact, require approval, deny, or quarantine) with hash-chained evidence, human approval workflows, and in-agent trust gates for LangChain, LangGraph, and Google ADK.
      30
      53 PyPI
      2
      MIT

    • agentloop

      luckeyfaraday
      A
      license
      Not graded
      quality
      C
      maintenance
      MCP server that enables AI agents to run a deterministic orchestration loop with decomposition, subagent execution, and review feedback across multiple LLM backends.
      61
      MIT
    Deploy Server
    One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

    TDQS

    A3.7/5.0

    Scored across 10 tools

    Disambiguation4/5

    Most tools have distinct purposes, but check_retrieval_drift and compare_runs both detect regressions, causing slight overlap. Descriptions help clarify, but an agent might confuse them.

    Naming Consistency4/5

    Most tools use verb_noun pattern (e.g., check_thresholds, evaluate_retrieval), but evaluate_rag_end_to_end is more verbose and deviates slightly. Overall consistent.

    Tool Count5/5

    10 tools is well-scoped for an evaluation server, covering retrieval, generation, comparison, and reporting without being overwhelming.

    Completeness4/5

    The set covers core evaluation workflows (retrieval, RAG, comparison, thresholds, reporting). Missing delete or batch management, but that's acceptable. The stub tool indicates a planned feature not yet implemented.

    Maintenance

    ActivityInactive
    ResponsivenessNo issues

    Resources