Skip to main content
Glama

eval_suite

Load evaluation templates, freeze baselines, run enterprise-specific eval suites, and query logs for AI agent auditing.

Instructions

企业专属 eval 套件(模板加载/基线冻结/运行/查日志)。

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
actionYes操作类型:instantiate=加载模板 / freeze=冻结基线 / run=运行评测 / query=查询日志
industryNo行业(instantiate 时选)
custom_casesNo自定义 case(instantiate 时可选)
enterprise_idYes企业 ID(必填)

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.1.0

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden of behavioral disclosure. It only restates the high-level operations already visible in the action enum and does not explain side effects, state changes, permissions, reversibility, or what happens after freeze/run operations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single compact sentence with a parenthetical list of operations, making it easy to scan. It contains no redundant wording, though the terse style sacrifices explanatory depth.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a multi-action tool with no output schema, yet the description does not explain which parameters apply to which action, expected results, log format, or success criteria. An agent would need to infer a lot from the action enum and parameter names, which is insufficient for correct invocation across all four modes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the input schema already documents all four parameters and their meanings. The description adds no parameter-level detail beyond the schema, which is acceptable under the baseline but does not enhance understanding of operation-specific parameter requirements.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies the tool as an enterprise-specific eval suite with four concrete operations: template loading, baseline freezing, running, and log querying. This is specific enough to distinguish it from generic evaluation tools like evaluate or evaluate_output, though it does not name those siblings explicitly.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus sibling tools such as evaluate_output, evaluate, or run_audit. The action enum implies possible uses, but the description offers no context, prerequisites, or exclusions to help an agent choose this tool over alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools