Add a test case with input and expected output to an evaluation suite, enabling dataset management for AI behavior analysis.
Add a case (input + expected) to an eval suite.
| Name | Required | Description | Default |
|---|---|---|---|
| body | Yes | CreateEvalCaseRequest as a JSON object string. Required: input_bundle (object). Optional: expected (any), metadata (object). Example: {"input_bundle":{"prompt":"hello"},"expected":"hi"} | |
| suite_id | Yes |
Changes observed during successful MCP inspections.
v0.3.0Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already disclose readOnlyHint=false and destructiveHint=false, so the description aligns with those but does not add any further behavioral context beyond stating the add operation. Given the openWorldHint=true, the description would benefit from noting idempotency, preconditions, or potential side effects, but none are provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a concise single sentence with no redundant words. It front-loads the verb and core resource, making it easy to process. It earns a 4 because while it is efficient, it could include a bit more context (such as clarifying the required suite_id) without much cost, but as-is it is well-structured.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple creation tool with few parameters and no output schema, the description is moderately complete. It covers the main action but omits important context like whether the eval suite must exist, how to obtain a suite_id, or how this tool compares to case-creation-from-run. These gaps prevent full completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description partially explains the body parameter as 'a case (input + expected)', which adds meaning to the JSON object schema. It does not add specific meaning to suite_id, though 'to an eval suite' implies the suite_id parameter is the target. With schema coverage at 50%, this adds some semantic value but does not fully compensate for the undocumented suite_id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb and resource: 'Add a case (input + expected) to an eval suite.' It is specific about the action and target resource, distinguishing this from generic case tools to some degree. However, it does not explicitly differentiate from related siblings like invariance_eval_case_create_from_run or invariance_case_create, so it misses the highest score.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no guidance on when to use this tool versus alternatives. It does not mention cases where the user should use invariance_eval_case_create_from_run, invariance_case_create, or other case-related tools. The purpose is implicit but there are no use-mode exclusions or alternative routing.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.