Server Quality Checklist

Tool Scores

• get_postal_code_metadataA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes what the tool returns (metadata including coordinates, region, fields) and implies it's a read-only lookup operation. However, it lacks details on error handling, rate limits, authentication needs, or data freshness. The description adds basic behavioral context but misses deeper operational traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and concise. It starts with the core purpose, lists key returns, provides usage guidelines, and details parameters and returns in labeled sections. Every sentence adds value without redundancy, making it easy to scan and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, but has output schema), the description is fairly complete. It explains purpose, usage, parameters, and returns. The output schema exists, so the description doesn't need to detail return values extensively. It could improve by addressing potential errors or sibling differentiation, but it covers the essentials for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains both parameters: 'postal_code: The postal code to look up' and 'country: ISO 3166-1 alpha-2 country code. Defaults to "US".' This adds meaning beyond the bare schema, clarifying the country code format and default value. It doesn't detail validation rules or examples, but covers the essentials.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get full metadata for a postal code including coordinates.' It specifies the verb 'Get' and resource 'postal code metadata' with key components like coordinates, county/municipality, and country-specific fields. However, it doesn't explicitly distinguish this from sibling tools like 'lookup_postal_code' or 'validate_postal_code', which likely have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance: 'Use this when someone needs geographic coordinates, timezone, or detailed location data for a postal code.' This gives context for when to select this tool. It doesn't explicitly state when not to use it or name alternatives among the siblings, but the guidance is sufficient for typical agent decision-making.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_postal_codeA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool checks validity, implying a read-only operation, but lacks details on error handling, rate limits, or authentication needs. It adds some context by specifying the default country, but more behavioral traits could be included.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the purpose, followed by usage guidelines and parameter details. Every sentence adds value, with no wasted words, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity, no annotations, and an output schema that covers return values, the description is mostly complete. It explains purpose, usage, and parameters well, but could benefit from more behavioral details like error cases or limitations, though the output schema reduces the need for return value explanation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains that 'postal_code' is 'The postal code to validate' and 'country' uses 'ISO 3166-1 alpha-2 country code' with a default of 'US,' clarifying semantics that the schema alone does not provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Check') and resource ('postal code'), and distinguishes it from siblings by focusing on validation rather than metadata retrieval or lookup. It explicitly mentions 'exists in a given country,' which differentiates it from tools like 'search_by_city'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'when someone wants to verify if a postal code is valid before processing an order, form, or address.' This clearly defines the context and distinguishes it from alternatives like 'get_postal_code_metadata' or 'lookup_postal_code,' which likely serve different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_cityA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the tool's core function and US-specific requirements but doesn't mention error handling, rate limits, authentication needs, or what happens with invalid inputs. The description adds basic context but lacks comprehensive behavioral details needed for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and front-loaded: purpose statement first, usage guidelines second, parameter explanations third, and return value last. Every sentence earns its place with no wasted words. The formatting with clear sections (Args, Returns) enhances readability without adding unnecessary length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, 0% schema coverage, no annotations, but the presence of an output schema, the description is quite complete. It covers purpose, usage, parameters, and return values adequately. The output schema handles return format details, so the description doesn't need to explain return structure. Minor gaps remain in behavioral transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing clear semantic explanations for all three parameters. It explains what each parameter represents, provides examples for 'city' and 'state', specifies when 'state' is required, and indicates the default value and format for 'country'. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Find') and resource ('postal codes for a given city'), distinguishing it from sibling tools like 'get_postal_code_metadata', 'lookup_postal_code', and 'validate_postal_code' which focus on different operations. The first sentence directly answers 'what does this tool do?' without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('when someone has a city name and needs the postal codes that serve it') and includes specific requirements for US cities (state parameter required). It distinguishes usage from sibling tools by focusing on city-based searches rather than metadata retrieval, lookup, or validation operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_postal_codeA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses behavioral traits such as the tool's international scope (works for 70+ countries) and default behavior (country defaults to 'US'), which are useful beyond basic functionality. However, it doesn't mention error handling, rate limits, or authentication needs, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose, followed by scope, usage guidelines, and parameter details. Every sentence adds value, with no wasted words, making it efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity, no annotations, and an output schema (which handles return values), the description is complete enough. It covers purpose, usage, parameters, and scope, providing all necessary context for an agent to use the tool effectively without redundancy.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds significant meaning beyond the input schema by explaining what 'postal_code' and 'country' represent, providing examples (e.g., '90210', 'US'), and noting the default for country. This fully documents the parameters in a helpful way.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'look up' and resource 'postal code', specifying it retrieves city, state, and region information. It distinguishes from siblings by focusing on lookup rather than metadata retrieval, validation, or city-based search, making it specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: 'when someone asks about a postal code, wants to know what city a ZIP code belongs to, or needs address information.' It also implies alternatives by listing sibling tools like get_postal_code_metadata, search_by_city, and validate_postal_code, though it doesn't name them directly, the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.