A
licenseB
qualityD
maintenanceAn MCP-style stdio server for evaluating AI agent outputs, enabling CI-friendly quality gates, regression comparisons, and canary promotion decisions.
3
MIT
dify-eval-mcp
Related Servers
Alternatives to dify-eval-mcp
No user-submitted related servers found.
Related Servers
Patronus MCP Serverofficial
AlicenseNot gradedqualityDmaintenanceEnables running LLM evaluations, experiments, and custom evaluators through a standardized MCP interface.16Apache 2.0
AgentMark MCP Serverofficial
AlicenseNot gradedqualityAmaintenanceMCP server for AgentMark. Exposes the full AgentMark API to AI editors: list traces and spans, manage datasets, write scores, run experiments, manage apps, deployments, environments, and annotation queues.352AGPL 3.0
multivon-mcpofficial
AlicenseAqualityAmaintenanceMCP server that gives AI coding agents direct access to evaluation tools.23Apache 2.0- FlicenseNot gradedqualityBmaintenanceMCP server for the Harbor hub that exposes evaluation jobs, trials, uploads, and published packages as tools for AI agents.-
- FlicenseNot gradedqualityBmaintenanceAn MCP server for prompt and RAG evaluation workflows, enabling users to register, create API keys, and run evaluations via Claude Desktop, Cursor, or other MCP clients, with a hosted backend for heavy evaluation work.1-
TDQS
A3.9/5.0
Scored across 5 tools
Disambiguation5/5
Each tool has a distinct, non-overlapping purpose: running evaluations, comparing runs, exporting/importing human reviews, and listing evaluation sets. The paired export/import tools are clearly separated by their direction of data flow, and run_eval vs list_eval_sets are unambiguously different (execution vs discovery).
Naming Consistency5/5
All tool names follow a consistent verb_noun pattern in snake_case: run_eval, compare_runs, export_review, import_review, list_eval_sets. The verbs are all lowercase and descriptive, and the nouns are appropriate. The slight singular/plural variation (eval vs eval_sets, runs) is semantically justified and does not break the pattern.
Tool Count5/5
Five tools is well-scoped for the domain of Dify evaluation workflows. Each tool covers a distinct step in the lifecycle (discover, run, compare, review), without unnecessary overlap or bloat. This is within the ideal 3-15 range and feels purpose-built rather than padded.
Completeness4/5
The tool surface covers the core evaluation workflow: listing eval sets, running them, comparing results, and handling human reviews. The only noticeable gap is the lack of a way to list or inspect existing run outputs, which an agent would need to know prior runs for comparison. This is a minor dead end that can be worked around if run artifacts are known or passed explicitly.
Maintenance
ActivityMaintained
ResponsivenessNo issues