Server Quality Checklist

Tool Scores

• verify_roundtripB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must carry the behavioral disclosure burden. It does reveal that the tool performs an encode/decode roundtrip and checks for data loss, which is meaningful. However, it gives no information about return values, success/failure signaling, side effects, or dependencies on the sibling tools.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core action. The second sentence adds a bit of context about the intended use, though it is somewhat generic and slightly redundant with the first sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, a verification tool should clarify what the caller receives on success or failure and how the verification is performed. The description only states intent, leaving the invocation contract under-specified for an agent to use confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single 'payment' parameter, and the schema already describes it as 'Payment data to verify'. The tool description adds no parameter-specific meaning beyond the general roundtrip concept, so the schema carries the semantic weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Verify') and a clear subject ('payment data survives encoding/decoding without loss'), which distinguishes it from the sibling generate/decode tools by emphasizing roundtrip integrity rather than encoding or decoding alone. However, it does not explicitly mention the generate-then-decode pipeline, so some disambiguation relies on the tool name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'Useful for testing data integrity' provides a clear general use case, implying when an agent might reach for this tool. However, it does not explicitly compare against generate_paybysquare, decode_paybysquare, or check_compliance, nor does it state when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_complianceA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It does state that the tool returns a detailed compliance report, which is useful, but it does not mention side effects, authorization requirements, error behavior, or any constraints beyond the obvious check operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences with no filler. It front-loads the core action and scope, and the second sentence adds meaningful return-value information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a validation tool, the description is adequate: it states the input, purpose, and return concept, and the nested payment schema is fully documented. However, with no output schema and no annotations, it would be stronger if it described the shape of the compliance report or what invalid results look like.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds the PayBySquare and banking standards context, but it does not add parameter-specific guidance beyond what the schema already provides; the 'detailed' parameter's meaning and default are already documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Check') and resource ('payment data compliance with PayBySquare and banking standards'), which clearly distinguishes it from sibling tools that generate, decode, or verify roundtrips. The first sentence alone lets an agent know what this tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The purpose implies the tool should be used when payment data needs validation against PayBySquare and banking standards, but the description provides no explicit when-to-use or when-not-to-use guidance. It also does not name or contrast alternatives like generate_paybysquare or decode_paybysquare.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• decode_paybysquareA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the disclosure burden. It clearly states the input format and that payment data is returned, but it does not explain error behavior, output structure, or any processing limitations. This is minimal but acceptable for a simple decoding tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence conveys the action, input format, and result without filler. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one simple parameter and no output schema, so the description should clarify what 'payment data' contains for downstream use. The current phrase is functional but vague; an agent may not know what fields or format to expect from the decoded data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents imageBase64 adequately. The description's reference to 'base64-encoded PNG image' adds no meaning beyond what the parameter schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states a specific verb ('Decode'), a clear resource ('PayBySquare QR code'), and the input format ('base64-encoded PNG image'). This distinguishes it from sibling generate_paybysquare, which creates rather than decodes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the verb and resource: the tool is for decoding a base64-encoded PayBySquare QR image. However, it does not explicitly mention alternatives like generate_paybysquare, check_compliance, or verify_roundtrip, nor does it say when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_paybysquareA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full disclosure burden. It explicitly discloses the main side effect—saving the QR code to a file—and the return behavior of providing a file path. It stops short of covering failure modes, overwrite behavior, or directory creation, so it is strong but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and free of filler: it opens with the main action, then lists the required and recommended fields, then states the output behavior. Every sentence earns its place and the structure is easy for an agent to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with a rich schema and no output schema, the description covers the essential guidance: what it generates, which fields matter most, and what it returns. Some gaps remain, such as not explicitly mentioning the required iban and not describing file-handling details, which prevents a perfect score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all parameters and defaults at 100% coverage, so the baseline is 3. The description repeats the required beneficiaryName and recommended swift, but it does not add substantial new meaning beyond the schema, and it omits that iban is also required within the payment object.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: generating a PayBySquare QR code PNG from payment data. This clearly distinguishes the tool from sibling tools such as decode_paybysquare, check_compliance, and verify_roundtrip, which serve different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives the core use case and highlights required and recommended fields, but it does not explicitly state when to select this tool over its siblings or when not to use it. Usage context is implied by the generation purpose rather than fully spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.