Server Quality Checklist

Tool Scores

• detect_card_typeA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the core behavior (uses the IIN/BIN prefix) and the input requirement (digits only, caller must strip spaces/dashes). However, it does not describe what happens for invalid inputs (e.g., unknown prefix, out-of-specified length) or clarify that it performs no validation (suggested by sibling). There is no mention of side effects, but none are expected. It provides some transparency but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences that pack essential information: purpose, network list, and input preprocessing instruction. There is no extraneous text. It front-loads the core action and follows with important input guidance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and an output schema, so description need not explain return values. It covers purpose, input format, and the IIN/BIN mechanism. It lacks explicit caveats (e.g., handling of unknown networks, validation not performed) but given complexity and annotations, it is reasonably complete. The lack of usage guidance vs. siblings is a minor gap, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds value beyond the schema: it clarifies the parameter is the full card number and explicitly states that spaces/dashes should be removed, which complements the pattern '^\d{8,19}$'. It also explains the number's use (IIN/BIN prefix) that the schema doesn't convey. However, it does not mention the parameter name by name or provide additional format details (e.g., maximum length, examples). Given the schema has strong constraints, the description provides marginal extra semantics but not complete coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Identifies the card network (Visa, Mastercard, American Express, Discover, Diners Club, JCB) from the card number's IIN/BIN prefix.' It specifies the resource (card number) and the action (detect card type), and lists the distinct networks. This distinguishes it from siblings like 'validate_card_number' (validation) and 'validate_iban' (IBAN), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes an input precondition: 'Accepts digits only (spaces/dashes should be stripped by the caller).' This is a clear usage instruction on formatting, but it does not provide guidance on when to choose this tool versus the sibling tools (validate_card_number, validate_iban). The usage is implied by the tool's name and purpose, but no explicit alternatives or exclusions are given. As such, it is adequate but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_card_numberA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It states the algorithm (Luhn) and input requirements (digits only, spaces/dashes stripped by caller), which is useful. However, it does not disclose what happens on invalid input (e.g., return value or error behavior) or any other side effects, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and every sentence adds value. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, clear algorithm) and the presence of an output schema, the description is fairly complete. It covers the algorithm, input constraints, and caller responsibility. It could mention the return format (boolean) but the output schema likely covers that, so this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It explains that the cardNumber should be digits only and that spaces/dashes should be stripped, which adds meaning beyond the schema's pattern. However, it does not explain the expected format beyond digits (e.g., length range) or the meaning of the return value, so it only partially compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Checks whether a card number passes the Luhn checksum algorithm.' It uses a specific verb ('checks') and resource ('card number'), and the mention of the Luhn algorithm distinguishes it from sibling tools like detect_card_type and validate_iban.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool (to validate a card number via Luhn) and includes a usage note about stripping spaces/dashes. However, it does not explicitly state when not to use it or mention alternatives like validate_iban for other validation needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_ibanA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With zero annotations, the description carries full burden and does add useful non-obvious behaviors: spaces are stripped and letters are case-insensitive. However, it omits other behavioral traits like side-effect-free execution, error behavior on invalid input, or whether validation is local. Adds genuine context but doesn't fully cover the space.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two efficient sentences: the first states purpose and validation depth; the second covers input normalization edge cases. Zero wasted words, information density is excellent, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter validation tool with an output schema, this description is nearly complete: it explains the validation logic (length + checksum) and accepted input forms. Annotations are absent, so a note on read-only/side-effect-free semantics would push it to 5, but the scope is fully appropriate for its complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate—and it does by explaining that the iban string tolerates spaces (stripped) and case variations. This meaningfully enriches the bare schema definition (string, 4-50 chars). Could go further by noting whether other whitespace (tabs) is stripped, but for one parameter it's well-handled.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Uses a specific verb ('Validates') with a well-defined resource ('International Bank Account Number (IBAN)') and names the exact validation standard (ISO 13616) and checksum algorithm (mod-97). The IBAN domain is self-evidently distinct from sibling card tools, and the technical specificity makes the tool's scope unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clarifies what 'validates' means by specifying both length checks and checksum verification, giving agents a clear sense of this tool's validation depth. However, it provides no explicit when-to-use vs. alternatives (e.g., no contrast with validate_card_number or note that this is local-only validation).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.