Server Quality Checklist

Tool Scores

• get_quality_rubricA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Mentions 'Free; takes no input.' but omits details like caching, authentication, or whether it updates. Adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two clear sentences: one stating action, another adding usage guidance and input status. No fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, description explains returns (rubric markdown plus version). Could elaborate on version meaning, but satisfies most needs for a simple fetch tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist; description confirms 'takes no input.' Details about the output (markdown and version) provide meaning beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states fetching the published Quality Gate rubric and version, distinguishing it from siblings like score_quality or verify_quality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises reading to understand what a quality score means and does not mean, implying when to use. Lacks explicit exclusions or alternatives, but context with sibling tools makes usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• certify_provenanceA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explains the cryptographic process (SHA-256 hash, Ed25519-signed receipt, timestamp), the nature of generator_metadata, and the payment requirement. It lacks details on error handling or response structure but provides solid behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and uses bold headers for key sections (PROVES, DOES NOT PROVE), improving readability. The payment note is integrated without bloating. A few redundant phrases could be trimmed, but overall it is well-structured and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains the returned receipt (Ed25519-signed) and its contents (hash, metadata, timestamp). It covers core functionality, limitations, and payment context. A more detailed return format would increase completeness, but it suffices for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds meaning beyond the schema by clarifying that generator_metadata is recorded verbatim and that the receipt only proves the claim, not the actual model. This provides valuable context for parameter interpretation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool certifies content origin via a provenance service, distinguishing it from siblings like verify_provenance. It specifies the action (certify), the resource (content provenance), and details what the receipt proves and does not prove, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly guides usage by explaining what the tool does and does not prove, preventing misuse. It also mentions the paid action note with x402 payments. However, it does not explicitly compare to siblings like verify_provenance or provide when-not-to-use scenarios, leaving slight room for improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_provenanceA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description fully explains the verification process (re-hashing content, checking Ed25519 signature, return shape with valid and details), including what valid means and how to interpret failures. It also clarifies that a 200 response does not guarantee validity. No annotations were provided, so the description carries the full burden and meets it excellently.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph that front-loads the action and includes all necessary details without redundancy. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite lacking an output schema, the description comprehensively covers the return structure and meaning. All parameters are clearly documented, and the tool's behavior is fully described, making it complete for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with clear parameter descriptions. The description adds value by explaining the verification process and linking the receipt parameter to certify_provenance, providing context beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies a provenance receipt against its content, with a specific verb 'verify' and resource 'provenance receipt'. It is distinct from sibling tools like certify_provenance (which creates receipts) and those for quality scoring, leaving no ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (after obtaining a receipt from certify_provenance) but does not explicitly state when to use or when not to use, nor mention alternatives. It provides no exclusions, so guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_qualityA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description fully discloses the verification process: re-hashing content, checking Ed25519 signature, handling forged/altered scores, return format ({valid, details}), cost (free), and HTTP response interpretation (200 means ran, always read 'valid'). No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each purposeful. Front-loaded with the core purpose, then technical details, then practical usage note. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a verification tool with no output schema, the description covers the return shape, HTTP response nuance, cost, and security mechanism. It is complete and self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining that receipt comes from 'score_quality' and detailing the verification mechanism, going beyond the schema's basic property descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool verifies a quality score receipt against its content using a quality-gate, with specific details about re-hashing and Ed25519 signature checking. It distinguishes from sibling tools like verify_provenance by focusing on quality scores.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some context (e.g., free, 200 response meaning) but does not explicitly state when to use this tool over alternatives like verify_provenance. No exclusions or alternate tool mentions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• score_qualityA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully carries the burden. It discloses that the output is an Ed25519-signed score receipt, explains the effect of target_score on receipt issuance and charges, and mentions potential micropayment requirements. No contradictions exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is clear and structured with sections (WHAT THIS IS, WHAT IT IS NOT, etc.), but is slightly verbose with some redundancy (e.g., repeating 'score content'). Front-loaded effectively with the main purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers key aspects: scoring dimensions, receipt signing, optional features, and payment notes. However, it does not fully detail the return structure beyond mentioning a signed receipt and breakdown. Given no output schema, slightly more specificity could help, but it is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds significant value beyond the schema. It explains that rubric_version must match the service's current version if provided, and describes the target_score's 'no pass, no pay' mechanism in detail, which is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool scores content against a published quality rubric, listing the four dimensions (clarity, completeness, internal consistency, obvious-error freedom) and the score range. It distinguishes itself from sibling tools like get_quality_rubric (which reads the rubric) and implies it is the scoring action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states what the tool is and is not, and explains the optional target_score parameter's 'no pass, no pay' behavior. It also notes the /v1/score endpoint is the paid action and may require micropayments, helping agents decide when to use the tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.