Server Quality Checklist

Tool Scores

• verify_receiptA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and does an excellent job. It discloses side effects ('read-only, no ledger writes, no file access beyond the in-memory argument, no network calls'), idempotency, and error behavior ('never raises on an invalid receipt, instead returns a normal result with is_error=true'). This far exceeds typical transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, moving from purpose to usage to side effects to error handling to parameters/returns. Every sentence carries essential information, and the formatting with separate paragraphs improves readability. It is comprehensive without being verbose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and only one parameter, the description must explain return values and error handling fully. It does so with explicit success/failure shapes and reason codes ('invalid_signature|hash_mismatch|unsupported_version|malformed_receipt'). It also covers prerequisites, side effects, and idempotency, making it complete for a fairly simple but security-relevant tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single receipt parameter, but the description enriches it significantly by specifying 'must include version, hash, signature, and the other receipt fields' and identifying it as the output of record_usage or the CLI. This adds structural expectations beyond the schema's generic object type, making the parameter's usage clearer.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Independently checks whether a single signed ComputeLedger receipt is authentic and untampered' which clearly states a specific verb, resource, and scope. It distinguishes from siblings by focusing on a single receipt, contrasting with verify_ledger's likely whole-ledger scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'Call this whenever you're handed a receipt... and need to confirm it's cryptographically valid before trusting the usage numbers inside it.' This gives clear when-to-use context, and it lists sources like record_usage and third-party receipts. However, it doesn't explicitly mention when not to use it or name alternative tools beyond sources, so it lacks full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ledgerA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses behavior: read-only, reads local file but never writes, no network calls, idempotent, returns empty list instead of error for missing ledger. This goes beyond what annotations could have provided and leaves no safety ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is structured with a clear opening purpose, a side-effects paragraph, parameter explanation, and return-value guidance. Every sentence serves a distinct informative purpose with no filler, and the most essential info is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description compensates by stating the return shape ('JSON array of full signed receipt objects') and listing the exact fields. It also covers error behavior, side effects, parameter details, and cross-tool usage, making it fully complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes the 'local' parameter at 100% coverage, but the description adds concrete file path details ('~/.computeledger/ledger.jsonl' vs './.computeledger/ledger.jsonl') and an equivalent CLI command. This enhances the schema's brief note, though the schema already covered the core meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Lists') and resource ('signed usage receipt recorded in the local ComputeLedger ledger'), plus a distinguishing scope ('in insertion order'). It clearly separates this tool from siblings like verify_ledger and verify_receipt by focusing on listing rather than verifying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to call it: 'to inspect or export local compute-usage history, e.g. before running verify_ledger, or to summarize usage across providers.' It also mentions alternatives ('Feed any single entry into verify_receipt, or call verify_ledger') and clarifies that missing/empty ledger is not an error.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• record_usageA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses side effects: it is 'mutating and NOT idempotent', appends a new entry to the local ledger, reads the private key from disk, makes no network calls, and returns is_error=true with a JSON error body on failure. This is far beyond what the schema alone provides.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured: opening purpose, usage guidance, side-effect disclosure, parameter summary, CLI example, and return-value explanation. Every sentence earns its place, and the most important behaviors are front-loaded in the first paragraph.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having no output schema, the description specifies the full receipt JSON structure (version, id, timestamp, prevHash, hash, signature, etc.) and explains verification via verify_receipt. It covers prerequisites, side effects, failure modes, and alternative tools, making it complete for a tool with this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is high (86%), but the description still adds valuable meaning beyond the schema: it provides concrete examples for provider/hardware ('aws', 'lambda-labs', 'nvidia-h100'), clarifies gpuHours/estimatedFlops as optional floats, and includes an equivalent CLI invocation that reinforces parameter semantics. The only minor gap is not elaborating on local beyond 'optional bool', but the schema already covers that.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource: 'Records one unit of compute usage ... as a signed, hash-chained receipt in the local ComputeLedger ledger.' It clearly distinguishes this tool from siblings by explaining it creates evidence of usage, while verify_receipt checks receipts and list_ledger does read-only lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to call ('after a workload finishes ... when you need durable evidence') and when not to ('Do not call it for read-only lookups (use list_ledger) or to check a receipt you already have (use verify_receipt)'). It also mentions a prerequisite (generate keypair) and failure behavior, giving clear context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_ledgerA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the transparency burden. It states 'Side effects: read-only, reads the local ledger file, no writes, no network calls, fully idempotent' and discloses stop-on-first-invalid behavior. It also explains that is_error is set when valid is false, which is a behavioral trait not visible in any schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although the description is longer than typical, it is front-loaded with purpose and then systematically covers usage, side effects, parameters, and return values. No sentence is wasted; each adds critical information that the schema and annotations do not provide.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter, no annotations, and no output schema, the description is exceptionally complete. It defines the exact return payload structure, the error flag behavior, and all edge cases (empty ledger, stopped verification). It fully compensates for the missing structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers the 'local' parameter with 100% coverage, so the baseline is 3. The description adds value by stating the default as false (while schema shows null), clarifying that it is the same meaning as in list_ledger and record_usage, and providing the equivalent CLI command with --local. This goes beyond what the schema alone offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource+scope: 'Verifies every entry's signature AND the unbroken hash-chain linkage across the entire local ledger in one call'. It clearly distinguishes from the sibling tool verify_receipt by explaining what it catches that a single-receipt check cannot.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'Call this for a full integrity audit of the local ledger before trusting exported totals, or periodically as a tamper-detection check.' It also mentions the prerequisite of an existing ledger and refers the reader to list_ledger, while the contrast with verify_receipt serves as an exclusion for when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.