Server Quality Checklist

Tool Scores

• decode_vinA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility. It discloses the paid nature ($0.002), the x402 payment mechanism, the fact that a challenge is returned without payment, and the VIN-only limitation. This goes beyond the minimum but does not cover rate limits or error behavior, which would be expected for a paid API.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences and front-loaded with the critical 'PAID $0.002' notice. Each sentence adds important information: the pricing/payment model, the core functionality and output examples, and the VIN-only restriction plus payment flow. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description must explain return values, and it does so by listing the validation and decode results. It covers the payment challenge behavior and the VIN-only scope. While it could mention error cases or rate limits, the provided context is sufficient for an agent to invoke the tool initially. A minor gap is not specifying the exact response format beyond examples.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides complete descriptions for both parameters (vin: 17-character VIN excluding I/O/Q; x_payment: base64 x402 payload forwarded as X-PAYMENT). The description adds examples of decode fields (make, model, year, etc.) but not new parameter-specific detail. Since schema coverage is 100%, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool decodes a Vehicle Identification Number and lists the specific outputs (17-char format, check-digit validation, full decode via NHTSA vPIC API with make/model/year/body class/engine/etc.). It distinguishes this from sibling validation tools (VAT, IBAN, email, geo lookup) through the unique resource and API.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly notes the tool is VIN-only and never converts plates to owners, which sets a clear boundary. It also explains the payment flow (without payment returns an x402 challenge; pass x_payment to settle), giving practical usage context. However, it does not explicitly compare against sibling tools or mention alternative VIN services, though the context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_vatA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the cost ($0.003, USDC on Base), the ~1h caching to handle flaky VIES, the return of company name/address, and the 402 challenge when unpaid. This is exceptional transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and front-loads key information (cost, purpose). Minor promotional language ('FLAGSHIP') and some redundant phrasing keep it from being perfectly concise, but every sentence contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main action, payment flow, and expected outputs (valid/registered, company info, or challenge). Although there is no output schema, the description adequately sets expectations for return values and behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents all three parameters with 100% coverage. The description adds minimal semantic value beyond restating the required inputs and the role of x_payment, so it meets the baseline without exceeding it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates EU/EEA VAT numbers with a specific verb and resource. It details the two-stage process (format/checksum pre-check plus live VIES lookup) and implicitly distinguishes it from sibling validators like validate_iban and validate_email.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on how to use the tool, including the payment prerequisite and the fallback to a 402 challenge. It does not explicitly name alternatives, but the domain-specificity makes the intended use obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_ibanA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure. It reveals the paid nature, the challenge flow, pure-offline behavior, and exactly what data is returned. This gives the agent a strong understanding of side effects and prerequisites.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three dense sentences with no filler. Front-loaded with payment warning, then clear purpose, then payment behavior. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a paid tool with no output schema, the description is remarkably complete: it explains purpose, inputs, outputs, payment/challenge flow, and offline nature. Enough for an agent to select and invoke correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameters with descriptions, so baseline is 3. The description adds context about the IBAN parameter (e.g., bank identifier extraction) and the x_payment parameter via the challenge flow, but does not significantly enhance parameter-level understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Validate') plus resource ('an IBAN') and enumerates concrete outputs: ISO 13616 length, mod-97 checksum validation, country, formatting, and bank identifier. This clearly distinguishes it from sibling tools like validate_vat or validate_email.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly describes the context (pure-offline, paid, returns x402 challenge without payment), but does not explicitly name alternative tools or state when not to use it. The sibling names make the intended use obvious, yet explicit exclusions are missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_emailA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility. It discloses the payment requirement, x402 challenge flow, statelessness, non-PII handling, no email storage, and no SMTP probing. This is comprehensive and gives the agent accurate expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-structured. It front-loads cost and purpose, lists checks, then covers statelessness and payment flow. Every sentence adds unique information, with no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a paid tool with no output schema and no annotations, the description is remarkably complete. It explains the challenge-response payment flow, the types of results returned, and the stateless/non-PII guarantee, giving the agent enough context to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with basic descriptions for both parameters. The tool description adds meaningful context about x_payment: that omitting it returns a 402 challenge and supplying it settles the call. This goes beyond the schema's definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'validate' and the resource 'email address', then enumerates the specific checks (syntax, MX, disposable, role). This distinguishes it from sibling validation tools (VAT, IBAN, VIN, geo) by domain and functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied: validating email addresses. It explicitly notes the tool does NOT probe SMTP mailboxes, providing a useful exclusion. However, it does not name alternative tools for mailbox checks or discuss when to choose this over other validators, so it falls short of 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• geo_lookupA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It explicitly states the tool is paid ($0.002), describes the x402 challenge that occurs without payment, and explains that passing x_payment settles the call. It also notes the bundled dataset has no per-day limits. This gives the agent a complete behavioral model.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loads the cost, and efficiently covers input formats, output details, and the 402/challenge/payment flow. There is no redundancy or fluff; every sentence contributes meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a lookup tool with no output schema, this description is complete: it lists all types of returned data (names, codes, hierarchy, population, coordinates), explains the two input modes, discloses the payment requirement and settlement process, and mentions dataset characteristics. The agent has all necessary context to invoke and interpret the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides 100% coverage for individual parameters, but the description adds critical cross-parameter semantics: it explains that the free-text 'query' can be used OR the structured 'country/state/city' fields, and clarifies the exact role of 'x_payment' in the payment flow. This goes beyond the schema's per-field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Resolve a place against a bundled open dataset' and lists specific outputs (ISO 3166-1/-2 codes, hierarchy, population, coordinates). This distinguishes it from sibling validation tools (VAT, IBAN, email, VIN) which operate in entirely different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (geographic lookups) and explains the payment workflow. However, it does not explicitly mention when not to use it or name alternatives among siblings. Since the sibling tools are for different validation tasks, the use case is self-evident, but explicit exclusions would earn a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.