List Hermes eval suites

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With annotations already declaring readOnlyHint=true, idempotentHint=true, and destructiveHint=false, the description reinforces these by explicitly stating the tool lists without executing ('without executing the harness'). It adds the specific scope of what gets listed (files, dry-run markers, case counts, harness path state), which is useful behavioral context beyond what annotations provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, focused sentence that compactly lists what the tool does and explicitly notes what it does NOT do. Every element earns its place with zero filler or redundant phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so the description needn't explain return values. The tool is a simple read-only listing operation with good annotations (readOnlyHint, idempotentHint) and no required parameters, so the description covers the essential behavioral contract. The main minor gap is no hint about what the parameters are used for, but for a straightforward listing tool with optional parameters, this is sufficient.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, but all three parameters (home, profile, toolkit_root) appear to be optional configuration path/profile selectors that are common across the hermes_* family. The description doesn't add syntax or format details for these parameters beyond what the schema provides, which is a gap since no parameter-level help exists. However, no parameters are required, and their role as environment selectors is reasonably inferable from context, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb+resource combination ('List configured Hermes eval suite files') and elaborates on what is included (dry-run markers, case counts, harness path state). It clearly distinguishes itself from sibling tools like hermes_eval_run and hermes_eval_start by explicitly noting it lists without executing the harness.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description conveys the 'when to use' by emphasizing this is an inspection-only operation that does NOT execute the harness, contrasting with evaluation-running siblings. It doesn't explicitly state when NOT to use it or name specific alternatives, but the context is clear enough for an agent to distinguish this read-only listing tool from eval-running tools.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.