Server Quality Checklist

Tool Scores

• check_vat_test_serviceC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'test VAT numbers' which implies this is for testing/diagnostic purposes rather than production validation, but doesn't specify what the tool actually does (e.g., returns validation results, service status, or test responses). No information about rate limits, authentication needs, or response format is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The bilingual format (English/Slovak) creates redundancy that doesn't add value for an AI agent. While the core information is present, the duplicate phrasing wastes space. The description is front-loaded with the essential purpose but could be more efficiently structured as a single clear statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a diagnostic/testing tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns (test results, service status, validation outcomes) or how to interpret results. Given the sibling tools suggest this is part of a VAT validation system, more context about this tool's specific role in testing versus production use is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage with clear enum values and explanations, so the baseline is 3. The description adds marginal value by emphasizing these are 'test' VAT numbers, reinforcing that this is for testing purposes rather than real validation. However, it doesn't provide additional context beyond what's already well-documented in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool tests VIES service integration with test VAT numbers, which is a clear purpose. However, it doesn't distinguish itself from sibling tools like 'check_vat_number' or 'check_vies_status' - it's unclear how this 'test service' differs from regular VAT checking operations. The bilingual phrasing adds some noise but doesn't obscure the core function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of when this test service should be used instead of 'check_vat_number' or 'check_vies_status', nor any indication of prerequisites or appropriate contexts. The agent must infer usage from the name and parameters alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_vies_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'check' implies a read-only operation, it doesn't specify whether this tool makes external API calls, has rate limits, requires authentication, returns real-time or cached data, or what happens when the service is unavailable. The description states what it does but not how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise - a single bilingual sentence that directly states the tool's purpose without unnecessary elaboration. The bilingual format adds minimal redundancy while serving accessibility. Every word contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter status-checking tool with no annotations and no output schema, the description is minimally adequate. It tells what the tool does but lacks important context about what information is returned (e.g., status codes, timestamps, member state details) and operational considerations. The agent knows the purpose but not what to expect from execution.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, maintaining focus on the tool's purpose. This meets the baseline expectation for parameterless tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: checking VIES service status and member state availability. It provides a specific verb ('check') and resource ('VIES service status'), though it doesn't explicitly differentiate from sibling tools like 'check_vat_test_service' which might also test service functionality. The bilingual nature (English/Slovak) doesn't affect clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_vat_test_service' or 'list_eu_member_states'. It doesn't indicate whether this is for operational monitoring, pre-validation checks, or troubleshooting. The agent must infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_vat_numberA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the VIES service but does not disclose behavioral traits like rate limits, authentication needs, response format, error handling, or whether this is a read-only operation. The description adds minimal context beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single bilingual sentence that efficiently conveys the tool's purpose. Every word earns its place, and it is front-loaded with the core functionality. No unnecessary details or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is minimal but adequate for a simple validation tool. It covers the purpose and service used, but lacks details on behavior, response format, or error handling. For a tool with 2 parameters and 100% schema coverage, it meets minimum viability but has clear gaps in transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear descriptions for both parameters (countryCode and vatNumber). The description does not add meaning beyond what the schema provides, such as explaining VAT number formats or validation rules. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Validate EU VAT number') and the resource/service used ('using VIES service'), with a Slovak translation reinforcing the purpose. It distinguishes from siblings by focusing on validation rather than testing service status or listing member states.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context for validating EU VAT numbers via VIES, but does not explicitly state when to use this tool versus alternatives like 'check_vat_test_service' or 'check_vies_status'. It provides clear scope (EU VAT validation) without exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_eu_member_statesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool's behavior as listing codes, which is straightforward, but does not disclose additional traits like rate limits, authentication needs, or output format. For a simple read-only tool with no parameters, this is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of a single, efficient sentence in two languages that directly states the tool's purpose. Every word earns its place, with no wasted information, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is complete enough for its purpose. It clearly explains what the tool does and its use case. However, it could be slightly enhanced by mentioning the output format (e.g., list of codes) to improve completeness, though this is minor.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description does not add parameter details, which is appropriate. A baseline of 4 is applied for zero parameters, as the description effectively covers the tool's scope without unnecessary parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('List') and resource ('EU member state codes'), and it distinguishes this from sibling tools by specifying it's for listing codes rather than checking/validating VAT numbers like the siblings (check_vat_number, check_vat_test_service, check_vies_status). The bilingual phrasing adds clarity without redundancy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool ('for VAT validation'), providing clear context. However, it does not specify when not to use it or name alternatives among the sibling tools, though the purpose differentiation implies usage scenarios. This is sufficient for clear guidance but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.