Server Quality Checklist

Tool Scores

• normalize_traceA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of disclosure. It reveals key behavioral details: coercion into a canonical shape, key-sorting for JSON, SHA-256 hashing, and the return of a normalized trace plus fingerprint. It does not mention error handling or mutation behavior, but these are less critical for a pure normalization function and the disclosed details are substantive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the primary action, then the return value and a rationale for the fingerprint. Every word earns its place, with no filler. The length is appropriate for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the core behavior and return value, which is essential given there is no output schema. It explains the purpose of the fingerprint (byte-comparison) and the canonicalization process. Missing details include error handling and relationship to sibling tools, but overall it is a self-contained and sufficient description for an agent to invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds conceptual meaning (e.g., 'coerce', 'canonical', 'key-sorted JSON') that enriches parameter understanding, but it does not annotate individual parameters beyond the schema. The schema already describes 'trace' as either an array or partial Trace, and input/output/model fields are self-explanatory. The description does not need to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Coerce' and clearly identifies the resource ('arbitrary tool-call trace') and the goal ('canonical agentsnap Trace shape'). It also distinguishes itself from sibling tools (validate_snapshot, diff_traces) by focusing on normalization and fingerprinting, not validation or diffing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage scenarios ('so two runs can be byte-compared cheaply') but does not explicitly state when to use this tool versus alternatives like diff_traces or validate_snapshot. No exclusions or preconditions are mentioned, leaving the agent to infer that normalization is a prerequisite for other operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• diff_tracesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It transparently explains the output behavior (same=true on structural match, otherwise lists additions/removals/changes) and how ignore_paths alters classification. It stops short of detailing error handling or side effects, but for a read-only diff tool the critical behaviors are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the primary purpose. It efficiently includes return value breakdown and an actionable example for ignore_paths without any filler. Every sentence contributes meaningful guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description compensates by fully explaining the return structure (same, additions, removals, changes). It also covers the optional ignore_paths parameter with examples. It lacks explicit notes on assumptions (e.g., traces must be normalized), but the tool name itself conveys that, and the description is otherwise thorough for a diff utility.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage, describing each parameter with types and meanings (e.g., baseline is 'typically loaded from a snapshot file'). The description adds value by giving a concrete example for ignore_paths ('fingerprint','tools[0].result_hash') and clarifying its prefix-matching behavior, but baseline/current semantics are already well-covered in the schema, so the description only marginally extends beyond it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'diff' and specific resource 'normalized agentsnap traces', distinguishing it from sibling tools like validate_snapshot and normalize_trace. It also explains the output categories (same, additions, removals, changes), leaving no ambiguity about its function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (comparing two traces) by explicitly naming the inputs and the purpose. It also gives practical guidance on using ignore_paths to drop noisy fields with a concrete example. While it doesn't explicitly contrast with sibling tools, the unique verb and resource make the usage context clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_snapshotA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It transparently discloses what it checks (required fields, tool-entry shape) and the return format (valid=true or list of issues). It does not mention side effects, but as a validation tool it inherently is non-mutating. It also does not detail edge-case behavior, but overall it is quite transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the action and target, and every sentence provides value. No redundant or unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a validation tool with a single parameter and no output schema, the description covers the purpose, validation criteria, and return format. It is sufficiently complete for an agent to understand the tool's behavior and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the schema already describes the snapshot parameter as a JSON-decoded object from expectSnapshot(). The tool description adds no extra parameter-specific details beyond what the schema provides, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates a snapshot object against the agentsnap Trace schema, specifying exactly what it verifies (required fields and tool-entry shape). It distinguishes itself from sibling tools normalize_trace and diff_traces, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides clear context for when to use the tool: to sanity-check a snapshot against the schema. However, it does not explicitly name alternatives or specify when not to use it, though the sibling tool names imply other operations like normalization and diffing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.