Server Quality Checklist

Tool Scores

• check_patchA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries full burden. It sufficiently discloses key traits: deterministic output ('Always `status: OK`'), no external dependencies (pure Python), and return of version. It does not mention error scenarios, but the simplicity warrants a high score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loads the primary action, and contains no redundant information. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no output schema), the description is mostly complete. It covers status and version but omits details on return format or potential error conditions. Slightly more detail could warrant a 5.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0 parameters, baseline is 4. The description adds value by explaining the tool's return content and lack of dependencies, but parameter semantics are inherently N/A.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Return server status + version') and distinguishes it from siblings like 'apply_patch' and 'restore_original' by emphasizing it always returns 'status: OK' and has no external dependencies, making it a safe read-only check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking server status and version, but it does not explicitly state when to prefer this tool over siblings like 'sha256_manifest' or in what contexts (e.g., before applying patches). The lack of explicit guidance lowers the score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• restore_originalA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but the description fully discloses the three-step process: compute hash, optionally verify, and overwrite target. It adds behavioral context beyond schema by explaining verification logic.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with summary, numbered steps, Args, and Returns. It is slightly lengthy but every element adds value, and the action is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  In the absence of an output schema, the description provides a complete return structure. All four parameters are covered, and the use case is fully explained with step-by-step logic.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description includes an 'Args' block that explains the purpose of each parameter, including optionality of expected_sha256 and the audit trail role of confirm_legal.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool copies original to restore_target and calls it a rollback primitive. This distinguishes it from siblings apply_patch (forward) and check_patch/sha256_manifest (verification/list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description identifies it as a rollback primitive, implying use for reverting patched files. It does not explicitly state when not to use or list alternatives, but context from siblings provides differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• apply_patchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description thoroughly discloses behavior: src is never modified, dst is created or overwritten, returns pre- and post-patch SHA-256, explains confirm_legal as audit trail (not enforced), and details return object keys. No annotations are present, so the description bears full responsibility and meets it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with sections (summary, args, returns, override-scope contract) and is front-loaded. It is slightly lengthy but every sentence adds value; minor conciseness improvements could be made.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is remarkably complete. It covers behavior, parameter semantics, return value structure, and policy constraints (override scope). It provides all necessary information for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining each parameter: src (source file, never modified), dst (destination file, created/overwritten), offset (0-based byte offset), new_bytes_b64 (base64-encoded bytes), confirm_legal (free-text justification). This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: copy src to dst, then splice new_bytes_b64 at offset. It is identified as the 'on-disk patch primitive', which distinguishes it from sibling tools like check_patch, restore_original, and sha256_manifest.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides context on when to use the tool via the 'Override-scope contract' section, explaining it is gated behind a policy override and restricted to Output/<run-id>/patches/. However, it does not explicitly state when not to use it or mention alternative approaches.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sha256_manifestA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden. It explains the tool computes a hash (read-only), describes the output format including path, size, and sha256, and notes its role in the patching process. No contradictions with annotations since none exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with docstring format (Args, Returns, usage note). Every sentence adds value: the core purpose, parameter definition, return structure, and use case. It is appropriately sized with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema) and context signals, the description covers all necessary information: input, output, and usage context. There are no gaps; it is complete for an agent to understand and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain the parameter. It documents 'path: file to hash' explicitly, adding meaning beyond the schema's type-only definition. The description covers the single parameter comprehensively.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool computes a SHA-256 manifest of a path, specifying the verb 'Compute' and the resource 'SHA-256 manifest'. It distinguishes from sibling tools (apply_patch, check_patch, restore_original) by explicitly tying into the patching workflow as a prerequisite step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to record the canonical SHA-256 of the original binary before any patch is applied', providing clear guidance on when to use it. It also connects to restore_original, indicating the hash is used as a rollback key, which helps differentiate from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.