Skip to main content
Glama

Server Configuration

Describes the environment variables required to run the server.

NameRequiredDescriptionDefault
DIFY_API_KEYNoYour Dify API key (app-xxx) for live mode. If not set, the server runs in mock offline mode.
DIFY_BASE_URLNoThe base URL of your Dify instance, used for live mode. If not set, the server runs in mock offline mode.

Instructions

Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.

This server publishes no instructions, or was last inspected before Glama recorded them.

Capabilities

Features and capabilities supported by this server

Protocol revision2025-11-25

CapabilityDetails
tools
{
  "listChanged": false
}
prompts
{
  "listChanged": false
}
resources
{
  "subscribe": false,
  "listChanged": false
}
experimental
{}

Tools

Functions exposed to the LLM to take actions

NameDescription
run_evalB

跑一批评测用例。mock=True(默认)离线运行,不需要 Dify 实例;live 模式用 DIFY_BASE_URL/DIFY_API_KEY 环境变量

compare_runsB

对比两次运行结果(竞品对比 / 版本回归)。输入是 run_eval 的 json_out 产物

export_reviewA

从运行结果导出人工盲评 CSV(自动判定留空,避免锚定效应)

import_reviewA

把人工评审 CSV 合并回运行结果;传多份 CSV 即众包(多数投票 + 一致率)

list_eval_setsA

扫描目录里的评测集(.yaml/.yml),返回名称、用例数、是否可 mock 离线跑

Prompts

Interactive templates invoked by user choice

NameDescription

No prompts

Resources

Contextual data attached and managed by the client

NameDescription

No resources

TDQS

A3.9/5.0

Scored across 5 tools

Disambiguation5/5

Each tool has a distinct, non-overlapping purpose: running evaluations, comparing runs, exporting/importing human reviews, and listing evaluation sets. The paired export/import tools are clearly separated by their direction of data flow, and run_eval vs list_eval_sets are unambiguously different (execution vs discovery).

Naming Consistency5/5

All tool names follow a consistent verb_noun pattern in snake_case: run_eval, compare_runs, export_review, import_review, list_eval_sets. The verbs are all lowercase and descriptive, and the nouns are appropriate. The slight singular/plural variation (eval vs eval_sets, runs) is semantically justified and does not break the pattern.

Tool Count5/5

Five tools is well-scoped for the domain of Dify evaluation workflows. Each tool covers a distinct step in the lifecycle (discover, run, compare, review), without unnecessary overlap or bloat. This is within the ideal 3-15 range and feels purpose-built rather than padded.

Completeness4/5

The tool surface covers the core evaluation workflow: listing eval sets, running them, comparing results, and handling human reviews. The only noticeable gap is the lack of a way to list or inspect existing run outputs, which an agent would need to know prior runs for comparison. This is a minor dead end that can be worked around if run artifacts are known or passed explicitly.

Maintenance

ActivityMaintained
ResponsivenessNo issues