Server Quality Checklist

Tool Scores

• context_receiptA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that it signs with HMAC-SHA256 using specific keys and that the receipt verifies with the same key. It does not describe return format, side effects, or error cases (e.g., missing key), but the core signing behavior is well explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise but slightly long—four sentences covering purpose, contents, signing mechanism, and verification. All sentences add value and are front-loaded with the core action. No wasted words, though it could be tighter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is moderately complex (signing, key management, verification), and the description covers the key aspects: what it signs, how it signs, and how to verify. It lacks an explicit return-value description (no output schema exists), but the purpose and integration with verify_context_receipt make the context complete enough for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not add meaning beyond the schema for the 'artifact' parameter; it mentions 'for this session' but does not elaborate on how to choose artifact values. This is acceptable given full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Issue') and resource ('HMAC-SHA256-signed context receipt'), listing exactly what it contains (files/views, SHA-256 hashes, delivery modes, token totals). It clearly distinguishes itself from siblings by describing the receipt generation role and pointing to 'verify_context_receipt' for verification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by framing it as 'the auditable answer to what did the AI see when it did this?' and mentions verification with a sibling tool. It does not explicitly state when not to use it or compare to context_stats, but the purpose is clear enough for an agent to select it for audit needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_context_receiptA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral disclosure burden. It discloses two meaningful traits: 'timing-safe' and 'canonicalized so JSON field order does not matter', which are security-relevant. It stops short of 5 because it does not describe the return value or failure behavior (e.g., boolean vs exception) for an invalid signature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that clearly states the action, inputs, and two important behavioral caveats without any filler or redundancy. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no output schema and no annotations, the description should ideally indicate what the verification returns or whether it throws on failure. The description omits this, but the security details and clear purpose provide enough context to push it above a minimal score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes the receipt parameter as 'exactly as returned by context_receipt', which is helpful. The signature parameter is only typed as string with no description. The tool description adds value by noting that canonicalization makes field order irrelevant for the receipt JSON, but it does not clarify the signature's format, encoding, or how it is obtained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Verify' against a 'context receipt + signature pair', clearly distinguishing it from sibling tools like context_receipt (creation) and read_file (reading files). Mentioning 'local receipt key' adds precision about the verification target.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The action 'Verify' clearly implies this tool is for validating a receipt+signature pair, so usage context is evident. However, it does not explicitly state when not to use it or mention alternatives such as context_receipt for creating receipts, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_filesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the lossless ledger behavior (full first, diff on re-reads) and per-file error handling without failing the batch. This gives agents a solid expectation of output and failure modes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, followed by behavioral details. Every clause earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains the key behaviors (ledger, error isolation) enough for an agent to predict the call's effect. Minor lack of explicit return format or ordering, but sufficient given the tool is a batch variant of read_file.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for both paths and force_full, so parameters are already well-documented. Description adds general batching context but no extra semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States clearly it reads a working set of text files in one call, with a specific verb and resource. Explicitly distinguishes from sibling read_file by contrasting 'one call for N files' vs 'N calls'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for batch reading multiple files and contrasts with the alternative of making N calls. Doesn't explicitly say 'use read_file for a single file' but the contrast implies it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• context_statsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It reveals methodology ('real tokenizer'), scope ('exactly what this server sent'), and the price assumption (LOSSLESS_PRICE_PER_MTOK, default $3/MTok). It does not explicitly state read-only safety, but the nature of stats reporting makes that implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with an engaging question, and each clause adds substantive detail (breakdown types, pricing, tokenizer). No filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter, no-output-schema tool, the description fully specifies what is reported and how it is computed. The agent can infer the return types (counts, USD amounts, file paths) and the scope (session file-read tokens) without ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the description need not explain parameters. It adds context about an environment variable (LOSSLESS_PRICE_PER_MTOK) which is relevant to the output, exceeding the schema's empty property list.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Where did this session’s file-read tokens go?' and enumerates specific outputs (totals, per-repo breakdown, heaviest files, dedup savings, USD estimate). This makes it distinct from the sibling tools, which focus on reading files or context receipts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: this tool is for analyzing file-read token consumption within the current session. It does not explicitly mention alternatives or when-not-to-use, but the purpose is self-evident and adequately guides selection among the listed siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• outlineA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the outline is 'cheap' and specifies the output (declaration lines, line numbers, bodies elided), implicitly indicating a read-only, non-mutating operation. It doesn't cover error handling or permissions, but provides strong behavioral detail for a simple tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core function followed by usage guidance. Every word earns its place; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter tool with no output schema, the description sufficiently explains the return value (declarations, line numbers, bodies elided) and the intended use case. It is complete for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a clear description for the single 'path' parameter. The tool description adds usage context but no additional parameter format or syntax details, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a structural outline of a file, listing declaration lines with line numbers and bodies elided. It distinguishes itself from sibling read_file by explicitly positioning itself as a pre-navigation step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: 'Use this to navigate a large/unknown file before reading specific parts with read_file.' This names the exact scenario and the alternative tool, making the conditional use clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_fileA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and exceeds expectations. It discloses the deduplication algorithm, lossless property, metering, signed receipt, and the exact semantics of force_full. It also clarifies that a later read returns either a marker or a diff, giving the agent a complete mental model of the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is somewhat long but information-dense. Every sentence adds value, from the initial purpose to the lossless guarantee and force_full option. The structure front-loads the core purpose, then explains edge cases, and ends with a concrete override—logical and easy to parse, though not as terse as the highest-caliber examples.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity and lack of an output schema, the description is thorough enough. It specifies return behavior for every scenario (full content, marker, diff) and mentions integration points like context_stats and signed receipts. The agent can confidently select and invoke this tool without needing further documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Though the schema already covers all four parameters, the description adds extra meaning: it explains that 'lines' is a 1-based inclusive range, 'symbol' uses heuristic extraction, and force_full overrides the diff mechanism. This goes beyond the schema's dry parameter names and enriches the agent's understanding of when each parameter is useful.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Read a text file through the context ledger.' It specifies the resource (text file) and the verb (read), and uniquely distinguishes itself from siblings like read_files by detailing the deduplication/diff behavior and optional symbol/line-range reads.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on behavior under different conditions (first read vs. later reads, unchanged vs. changed views) and explains when to use options like symbol, lines, and force_full. However, it does not explicitly name alternative tools or state when not to use this tool, so it stops short of full exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.