Server Quality Checklist

Tool Scores

• search_countriesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description must disclose behavior. It states it returns matching countries but lacks details on error handling, case sensitivity, or whether criteria values are exhaustive. Minimal beyond basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy. The first sentence immediately states the core function and criteria, meeting the front-loading principle.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search with one parameter and an output schema, the description is adequate but leaves ambiguity on criteria meaning (e.g., 'easiest' definition). Could more explicitly state the return is a list of country objects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description explains the criteria parameter by listing three meaningful values, adding significant context. However, it does not clarify if these are the only allowed values or provide format constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool searches countries by specific criteria (no_quarantine, easiest, titer_test), differentiating it from sibling tools that check policies or compare airlines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this vs sibling tools like check_country_requirements or compare_airlines. The description implies a general search use case but does not specify exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_country_requirementsA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden of behavioral disclosure. It states what the tool returns but does not mention side effects, authentication requirements, rate limits, or whether it modifies any state. The fact that it is a 'check' suggests a read-only operation, but this is not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with the first sentence stating the core purpose and example input, and the second enumerating the output types. Every word contributes clear information with no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description need not detail return values. It covers the main inputs and outputs well. However, it could be more complete by mentioning the optional direction parameter and its meaning, especially since the schema has no descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% coverage (no parameter descriptions), so the description must add meaning. It provides examples for country_code (US, FR, JP, AU), which adds value. However, it does not explain the direction parameter (e.g., import vs. export) or its default value, leaving a gap in understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Check pet import/export requirements'), specifies the input method ('by ISO code'), provides examples (US, FR, JP, AU), and lists the output types (vaccination, microchip, health certificate, quarantine, permit requirements). This distinguishes it from sibling tools like check_airline_policy and compare_airlines, which focus on different aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for checking pet import/export rules for a country, but does not explicitly state when to use this tool versus alternatives like check_airline_policy or search_countries. It lacks exclusion criteria or contextual cues that would help an agent decide between siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_airlinesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the output as a ranked list but lacks details on permissions, rate limits, or edge cases (e.g., no matches). The behavior is minimally transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the purpose and criteria, the second describes the output. No unnecessary words, information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (one parameter, no nested objects, existing output schema) and the context of sibling tools, the description provides sufficient information for an agent to invoke the tool correctly, though it could mention that the output is a list of airline names.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage and only a generic string type for 'criteria'. The description adds significant meaning by enumerating the allowed values and their definitions (e.g., 'cabin' means allow cabin pets), which helps the agent select appropriate inputs.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'compare' and the resource 'airlines', specifying the exact criteria available ('cabin', 'cargo', 'brachycephalic', 'cheapest'), which distinguishes it from sibling tools like check_airline_policy.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lists the criteria for comparison, implying when to use this tool (when needing a ranked list based on specific pet-related features). However, it does not explicitly state when not to use it or mention alternative tools, but the context signals from sibling names provide some differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_coverageA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully covers the behavior: a read-only operation to get coverage counts. It is clear and has no hidden side effects or disclaimers. Could mention if the data is real-time or cached, but not required for simplicity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence with no wasted words. The key information is front-loaded and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no parameters and an output schema, so the description is complete for its low complexity. It adequately informs the agent of the tool's purpose and return value.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so the description naturally adds meaning beyond the empty schema. It explains what the tool returns, fulfilling the role for parameter-less tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Get' and clearly states the resource 'Pawgo's data coverage' and what it provides: counts of airlines and countries. It is distinct from sibling tools like 'check_airline_policy' which focus on specific items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving overall coverage statistics but does not explicitly state when to use it versus alternatives or when not to use it. Given siblings, more guidance would help.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_airline_policyA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It implicitly indicates a read-only query ("check") and lists return data, but does not explicitly state it is non-destructive or mention auth/rate limits. This is sufficient for a simple lookup tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no extraneous information. The first sentence states the action and parameter format; the second lists return values. Front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema exists), the description fully covers its functionality and return values. No gaps remain for the intended use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining the parameter: an IATA code with examples (AA, DL, LH, AF). This adds meaning beyond the bare schema definition, though it could specify case sensitivity or format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks pet travel policies for a specific airline by IATA code, specifying return fields (cabin/cargo policies, breed restrictions, etc.). This distinguishes it from sibling tools like check_country_requirements or compare_airlines, which cover different domains.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use when you need airline-specific pet policies. It does not explicitly discuss when not to use or mention alternatives, but the purpose is straightforward and the sibling tool names imply distinct functions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.