Run evaluation suites to measure prompt variant performance, providing scores and pass/fail results to guide prompt optimization.
Run an evaluation suite from the Eval Framework. Executes test cases against a prompt variant and returns scores and pass/fail results.
| Name | Required | Description | Default |
|---|---|---|---|
| suite | Yes | Name of the eval suite to run | |
| variant | No | Prompt variant to evaluate | default |
Changes observed during successful MCP inspections.
v1.0.0Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate non-read-only, idempotent, non-destructive behavior. The description adds that it executes test cases and returns scores and pass/fail results, which is useful context. It does not disclose potential side effects such as external model calls, result storage, or runtime cost, but the annotations cover part of this burden.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no filler. It leads with the primary action, then expands with the execution and output behavior, making it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one required parameter and no output schema, the description covers the essential input and output behavior: it runs test cases and returns scores and pass/fail results. It does not explain error cases or whether results are persisted, but given the annotations and low complexity, it is reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the schema already fully documents 'suite' and 'variant'. The description loosely mirrors those meanings by mentioning 'evaluation suite' and 'prompt variant', but adds no additional detail about formats, defaults, or value constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Run') and a specific object ('an evaluation suite from the Eval Framework'), then clarifies the action with 'Executes test cases against a prompt variant'. This clearly separates it from the sibling tools, which are about document search, system status, and prompt retrieval.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes the main use case obvious: run an eval suite and see pass/fail results. However, it does not explicitly state when to choose this tool over alternatives, nor does it mention prerequisites such as the suite existing or which variant is appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.