Server Quality Checklist

Tool Scores

• validate_isbnA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden. It discloses that it checks the 978/979 prefix and mod-10 check digit, and returns the expected check digit on failure. This conveys the tool's behavior adequately.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with 'USE THIS'. Every word is functional; no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description explains the return behavior (returns expected check digit on failure). Could mention success return, but sufficient for a validation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers the isbn parameter with description. The description adds 'hyphens and spaces are ignored,' which provides input handling detail beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'verify an ISBN-13 book identifier' and explains the checks performed (prefix and check digit). It distinguishes from siblings like validate_card, validate_iban, validate_vin which target different identifiers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'USE THIS to verify an ISBN-13 book identifier instead of trusting that 13 digits are correct.' Provides clear context for when to use. Does not explicitly state when not to use, but sibling tools indicate distinct use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_ibanA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes validation logic (country, length, checksum) and return values (country, check digits, BBAN). No annotations provided, so description carries full burden; it is thorough but does not mention external dependencies or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words, front-loaded with directive 'USE THIS'. Every sentence contributes meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description adequately explains what is returned. Single parameter case; no gaps. Complete for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes the 'iban' parameter; the description adds that spaces are ignored, providing additional clarity beyond the schema. With 100% schema coverage, this extra detail is valuable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it validates IBANs, specifies what is checked (country, length, checksum), and explicitly distinguishes from sibling tools that validate cards, ISBNs, and VINs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs when to call ('whenever a user supplies a bank account for a payment, payout or invoice'). Does not mention when not to use or alternatives, but siblings are clearly different domains.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_cardA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavioral traits: it performs Luhn checksum, brand detection, and length verification, but explicitly states it does not check card existence, activity, or funds. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is two sentences: the first commands usage and sets context, the second specifies what it does and does not. Extremely concise and front-loaded with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and one parameter with full schema coverage, the description is remarkably complete. It covers purpose, actions, exclusions, and parameter behavior adequately for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers the single parameter 'number' at 100% with description of ignoring spaces/dashes. The description adds value by explaining the validation operations performed on that number, which goes beyond the schema's parameter description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it validates a payment card number's structure, Luhn checksum, brand detection, and length. It distinguishes from siblings like validate_iban, validate_isbn, and validate_vin by targeting payment cards specifically.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to use this tool before using a card number, advises against assuming validity or guessing brand, and clarifies what it does NOT check (real, active, funds). However, it does not mention when to use alternatives or when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_vinA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: checks allowed alphabet (no I/O/Q) and ISO 3779 check digit in position 9, and returns expected check digit on failure. No hidden traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no wasted words. First sentence gives purpose and usage context, second explains checks and output. Front-loaded with critical guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple validation tool with one parameter and no output schema, the description is largely complete. It explains what it checks and output on failure, though success output (presumably boolean) is implied but not stated.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, but the description adds significant meaning beyond 'The 17-character VIN to validate' by explaining validation rules and output behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates a vehicle VIN before acting on it, with a specific verb ('verify') and resource ('VIN'). It distinguishes from sibling tools (validate_card, validate_iban, validate_isbn) by focusing on VINs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises to use before acting, warns against assuming a 17-character string is valid, and describes what checks are performed. Provides clear context but does not explicitly mention when not to use or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.