Server Quality Checklist

Tool Scores

• incident_timelineA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint, idempotentHint, and destructiveHint, but the description adds crucial context: 'read-only, stateless, no network — nothing is executed.' It also discloses the error behavior for malformed events, returning '{error, hint}' with the offending index, which goes beyond the annotation set.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is organized into labeled sections (WHEN, INPUT, RETURNS, GOTCHAS) that allow quick scanning. Every sentence adds substantive value—no filler or redundancy—while front-loading the core purpose. The length is justified by the tool's complexity and the need to cover invocation context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description fully specifies the return object structure (event_count, window, spikes, hypotheses, next_checks) and mentions nested elements like suggested_check. It covers all four parameters, usage prerequisites, and error handling, making it complete for an agent to decide when to use it and how to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description carries full responsibility for explaining parameters. It specifies the 'events' envelope fields, ts formats (ISO-8601, epoch seconds/millis), normalized severity, and explains all optional parameters (bin_seconds, z_threshold, top_n) with defaults and behavior. This far exceeds the bare schema and fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Correlate already-fetched VMware events into one incident view,' which clearly states the verb (correlate), resource (already-fetched VMware events), and outcome (one incident view). It distinguishes itself from the sibling 'list_symptom_categories' by explicitly stating it does not fetch anything itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The WHEN section gives explicit use-after-fetch guidance, names the data-source skills that produce the required input, and directs users to 'list_symptom_categories' when unsure which events to pull. It also states what the tool does NOT do ('does NOT fetch anything itself') and mentions remediation routing to vmware-aiops/vmware-pilot, covering both preconditions and follow-ups.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_symptom_categoriesA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly, idempotent, not destructive), the description adds rich behavioral context: it notes no network access, describes the return envelope and item shape, and discloses that the routing table is a fixed constant so truncated is always false and total exact. This goes well beyond what annotations alone convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Every sentence contributes a distinct piece of information: purpose, usage trigger, follow-up action, output structure, and behavioral guarantees. There is no redundant or filler content, and the description is well-organized for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description fully specifies the return envelope and item fields, along with edge-case behavior (truncated always false). It also covers integration with the sibling tool and the use case, making it complete for a simple parameterless tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes no parameters, and the description explicitly states 'Takes no parameters.' This is fully consistent with the empty input schema, leaving no ambiguity. The baseline for 0-param tools is 4, and the explicit statement plus the lack of any parameter needed earns a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists symptom categories with example keywords and suggested next check. It uses a specific verb ('List') and resource ('symptom categories') and differentiates itself from the sibling tool incident_timeline by explicitly directing the user to pass gathered events to that tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit usage context: 'Use this when you don't yet know what to look at' and explains it turns 'something's wrong' into concrete investigation steps. It also names the follow-up action ('pass them to incident_timeline'), covering both when and how to use it relative to the alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.