Server Quality Checklist

Tool Scores

• validate_ibanA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses the validation algorithm (ISO 7064 mod-97) and the extra Portuguese IBAN check, which is useful. However, with no annotations provided, it does not describe the return format, error behavior, or edge cases, so the description only partially carries the transparency burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short, front-loaded sentences with no filler. Every word adds useful information about the tool's purpose and behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one well-documented parameter, but without an output schema the description should clarify what the tool returns (e.g., boolean, status object, or detailed results). The phrase 'Indica também' hints at output but does not specify its form, leaving some incompleteness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already covers 100% of the single parameter with a clear description ('IBAN a validar, com ou sem espaços'). The description adds no additional parameter-specific meaning, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Valida') with a clear resource (IBAN) and adds the checksum standard (ISO 7064 mod-97) and the Portuguese-specific validity check. This clearly distinguishes it from sibling tools like validate_nif and validate_postal_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It is implicitly clear that this tool is for validating IBANs, and the sibling tool names provide context for alternatives. However, the description does not explicitly state when to use it versus other validation tools or mention any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_postal_codeA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the format validation but does not disclose what the tool returns (e.g., boolean, throws exception) or behavior on invalid input. This is a significant gap for a validation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that directly states the tool's purpose without any filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one parameter and no output schema. The description explains the input format but omits the output behavior, which the agent needs to know since no output schema is provided. However, overall it is reasonably complete for a basic validation tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents the single postalCode parameter with an example. The description adds the formal pattern 'CP7, NNNN-NNN', which provides additional semantic detail about the expected format, enhancing understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: validating a Portuguese postal code format, with a specific pattern (CP7, NNNN-NNN). This is a specific verb+resource, distinguishing it from sibling tools like validate_nif and validate_iban, which handle different entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives. It implicitly suggests use for postal code validation, and the sibling tools are for other validation targets, but there is no explicit comparison or excluded scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_nifA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses the validation checks and the output (taxpayer type), but does not state behavior for invalid NIFs (e.g., error vs. false return) or any side effects. This is acceptable for a validator but leaves some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the main action (validates a NIF) followed by specific details. No waste, easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (1 parameter, no output schema), the description covers the core behavior and return value. It could be more explicit about error handling, but for a validation tool the described behavior is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides full coverage of the single parameter 'nif' with examples and format. The description adds no additional parameter semantics beyond what the schema provides, so the baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to validate a Portuguese NIF, checking structure, initial digit, and control digit, and return the taxpayer type. This distinguishes it from sibling validators like validate_iban and validate_postal_code.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the usage context (validating Portuguese NIFs) but does not explicitly mention when to choose this tool over alternatives or provide exclusion criteria. For a simple validator with obvious purpose, this is adequate but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_holidaysA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the scope (includes movable holidays based on Easter) but does not mention return format, error handling, side effects, or whether the operation is read-only. For a simple listing tool this is acceptable but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that is front-loaded with the main action and resource. Every word earns its place, with no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple nature (1 parameter, no output schema), the description covers the purpose but omits details about the return value structure (e.g., whether it returns dates, names, or objects). Without an output schema, the description should explain the output shape to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the only parameter 'year' has a clear description. The tool description adds value by explicitly noting that the list includes movable holidays based on Easter, which explains why the year is required and how it affects results beyond the schema's basic type/range information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Lista' – lists) and a precise resource ('feriados nacionais obrigatórios de Portugal' – mandatory national holidays of Portugal) for a given year. It also distinguishes itself from siblings like is_holiday by focusing on a full yearly list rather than a single date check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (for a given year) but does not explicitly state when to prefer this over alternatives, nor does it mention any exclusions or contrasting sibling tools. The context is self-evident for listing holidays, but the guidance is implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• is_holidayA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of disclosing behavior. It states the tool 'indicates' whether a date is a holiday, which implies a boolean response. It does not detail error handling for invalid dates or the exact output format, but for a simple predicate tool the core behavior is fairly transparent. The description is not misleading and provides enough to form a mental model.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, scannable sentence in Portuguese that conveys all essential information. There is no redundancy or filler, and it front-loads the core purpose. Every word earns its place, making it appropriately sized for a simple query tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (one parameter, no output schema, no annotations), the description is nearly sufficient. It explains the scope (national holiday in Portugal) and the parameter format. However, it does not explicitly state the return type (e.g., boolean) or behavior for invalid dates, which would be helpful given no output schema exists. Still, the simplicity of the tool means the description is close to complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage for the single 'date' parameter, including its format (YYYY-MM-DD) and pattern. The description repeats the format in parentheses but adds no additional meaning beyond what the schema already provides. This aligns with the baseline of 3 for high schema coverage, as the parameter semantics are fully handled by the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Indicates whether a date (YYYY-MM-DD) is a national holiday in Portugal.' It uses a specific verb ('indicates'), specifies the resource (a date), and scopes it to Portugal. This distinguishes it from the sibling validation tools (validate_nif, validate_iban, validate_postal_code) and list_holidays, which either validate identifiers or list holidays rather than checking a specific date.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for checking a single date's holiday status, providing clear context for when to use it. However, it does not explicitly compare itself to list_holidays or say when not to use it, though the distinction is fairly obvious. Since there are no explicit exclusions or alternatives mentioned, it falls short of a 5 but is clear enough for a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.