Server Quality Checklist

Tool Scores

• export_audit_bundleA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It explains that the bundle is signed, includes a tamper-evidence report, flags sequence gaps and hash-chain breaks, and defaults to exporting everything when no dates are provided. This gives meaningful insight beyond a simple 'export' phrase, though it could still clarify whether the operation has side effects (likely none).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the primary purpose, followed by two sentences of essential detail. Every word adds value with no redundancy or vagueness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only two optional parameters and no output schema, the description covers the main context: purpose, security features, and parameter behaviors. It might lack details about the exact delivery format of the bundle, but the description is otherwise adequate for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage by describing 'from' and 'to' as ISO 8601 timestamps for issued_at. The description adds further value by reinforcing the ISO 8601 format and explicitly stating the behavior when both are omitted (export everything).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Export all receipts in a date range as a signed audit bundle') and names the intended audience (compliance officer or auditor). This distinguishes it from sibling tools like issue_receipt and verify_receipt, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (for compliance/audit purposes) and explains parameter behavior (date range, omit both for everything). It does not explicitly name alternatives or exclusions, but the sibling tools are sufficiently distinct that the intended usage is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_receiptA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool validates signature, schema, and payload-hash format, and reports whether the receipt matches the vault copy. This gives meaningful behavioral insight beyond the generic name, though it doesn't mention potential error conditions or network/access dependencies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two succinct sentences: the first states purpose and key details, the second gives usage guidance. No redundant or filler content; the most important action and resource appear immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with no output schema and no annotations, the description covers what the tool does, the specific checks, and when to use it. It doesn't explicitly state the return format, but the phrase 'reports whether' implies a boolean result, which is adequate given the simplicity of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides a baseline by describing 'receipt' as the full receipt JSON object. The description adds semantic value by specifying the receipt type ('agent-receipts/v1') and the specific verification checks performed, which enriches the meaning of the parameter beyond the schema's minimal description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Check') and resource ('agent-receipts/v1 receipt'), and lists concrete validation details (Ed25519 signature, schema, payload-hash format, vault match). This clearly distinguishes it from sibling tools like issue_receipt, which creates receipts, and export_audit_bundle, which exports audit data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'Use this before trusting a receipt presented by another agent or system.' It provides clear context but doesn't explicitly mention when not to use it or cite alternatives beyond the sibling list, though the siblings are not reasonable substitutes for verification.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_service_infoA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It states exactly what is returned (manifest components) and the purpose, making the behavior predictable. The read-only nature is implied by 'Return', and no side effects are relevant for a getter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that packs specific details (schema version, signing key id, key format, purpose) without any fluff. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter tool with no output schema, the description fully specifies the return contents and intended usage. There are no gaps that would prevent an agent from selecting and invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description need not explain parameter usage. The baseline of 4 for no-parameter tools applies; the description adds value by clarifying the output format without needing parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Return' plus the resource 'service manifest' and enumerates the exact contents: receipt schema version, signing key id, and Ed25519 public key. It clearly distinguishes from sibling tools (issue_receipt, verify_receipt, export_audit_bundle) by focusing on service metadata rather than receipt operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'to pin for offline verification' provides clear context for when to use the tool. It does not explicitly name alternatives or exclusion criteria, but the use case is evident and the tool's purpose is unique among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• issue_receiptA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses that payloads are SHA-256 hashed and immediately discarded (never stored), and that the returned receipt verifies offline. This goes beyond a minimal description and provides key behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose and no filler. Every sentence provides actionable information, from what the tool does to when to call it and what it returns.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 7 parameters and no output schema, the description covers what the tool does, when to use it, payload privacy, and the return value. It is sufficient for an agent to invoke correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters, including the payload hash-and-discard behavior, so the description adds little beyond what the schema already states. The mention of 'action details' is generic and does not enrich parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Create a cryptographically signed, timestamped receipt proving that an agent performed an action.' This clearly distinguishes from siblings like verify_receipt and export_audit_bundle by focusing on receipt creation rather than verification or export.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Call this immediately after any consequential action' and gives concrete examples (approving an invoice, validating a supplier). It does not explicitly mention when not to use or name alternative tools, but the context is strong enough to guide the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.