Discover .ia-eval.yaml LLM test suite files in the project directory. Scans CWD and standard sub-directories (evals/, tests/, contracts/). Returns file paths ready to pass to run_eval_contract.
| Name | Required | Description | Default |
|---|---|---|---|
| dir | No | Directory to scan (defaults to server CWD) |
| Name | Required | Description | Default |
|---|---|---|---|
| dir | No | ||
| count | No | ||
| files | No |
Changes observed during successful MCP inspections.
Input schema / examplesAdded value: +[
+ {}
+]Output schema / propertiesAdded value: +{
+ "count": {
+ "type": "number"
+ },
+ "dir": {
+ "type": "string"
+ },
+ "files": {
+ "items": {
+ "type": "string"
+ },
+ "type": "array"
+ }
+}Output schema / (root)Previous value: -nullNew value: +{
+ "additionalProperties": true,
+ "type": "object"
+}Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so safety is covered. The description adds useful context beyond that: it scans CWD plus evals/, tests/, contracts/ and returns paths – specifics the annotations do not convey. Minor gaps on recursion depth or hidden-dir handling keep it at 4.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three tight sentences, front-loaded with the resource and pattern, then scan scope, then output purpose. No wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Output schema exists, so return values needn't be explained, and annotations cover the safety profile. The description fully equips an agent to call this for file discovery before run_eval_contract.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The single dir parameter has 100% schema description coverage ('Directory to scan, defaults to server CWD'), so the schema already carries the semantics. The description only implies scanning behavior without adding syntax, path format, or override specifics, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (discover) and resource (.ia-eval.yaml LLM test suite files) with the exact file pattern, and explicitly names the sibling run_eval_contract as the consumer so an agent can distinguish the discovery step from the execution step.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'Returns file paths ready to pass to run_eval_contract' clearly implies the find-then-run workflow and positions this against that sibling. It doesn't explicitly state when NOT to use it or other alternatives, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Add one secure layer between your agents and this server.