Server Quality Checklist

Tool Scores

• get_flight_option_detailsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes a read operation ('retrieve'), but doesn't disclose behavioral traits such as whether it requires authentication, has rate limits, what happens if parameters are invalid, or the format of the returned data. The description adds minimal context beyond the basic purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded with the main purpose. It avoids redundancy but could be slightly more concise by removing the slightly repetitive second sentence about 'detailed information'.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete. It doesn't explain what the return values look like (e.g., structure of segments, price details), potential errors, or other behavioral aspects needed for effective tool use. For a tool with 2 parameters and no structured output, more context is warranted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters (search_id and offer_id) with descriptions. The description mentions 'search results' and 'specific flight option', which aligns with the schema but doesn't add significant meaning beyond it. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('retrieve detailed information') and resource ('specific flight option from the search results'), and mentions specific data elements like segments, price, and baggage info. However, it doesn't explicitly differentiate from sibling tools like 'get_flight_options' beyond mentioning 'detailed information' versus presumably summary information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by stating it's 'useful for getting more granular information about a specific flight option' and references 'search results' from a previous search. However, it doesn't explicitly state when to use this versus alternatives like 'get_flight_options' or 'request_booking_link', nor does it provide exclusions or prerequisites beyond the implied need for a search_id and offer_id.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_booking_linkB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'generates a booking link' but lacks details on what this entails—e.g., whether it initiates a booking process, requires user authentication, has rate limits, or returns a temporary/permanent link. For a tool with no annotations and potential side effects (generating links), this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that are front-loaded: the first states the purpose, and the second provides usage context. There's no wasted text, but it could be slightly more structured (e.g., bullet points for clarity).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a tool that likely has behavioral implications (generating booking links), the description is incomplete. It lacks details on what the tool returns (e.g., URL format, expiration), error conditions, or prerequisites beyond the parameters. This leaves gaps for an AI agent to understand the full context of use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters (search_id, offer_id, agency_id) with clear descriptions. The description adds no additional meaning beyond what the schema provides, such as explaining how these IDs interrelate or their format. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'generates a booking link for a specific flight option' (verb+resource). It distinguishes from sibling tools like search_flights and get_flight_options by focusing on booking link generation rather than searching or retrieving details. However, it doesn't explicitly differentiate from get_flight_option_details beyond the booking link aspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'after the user expressed intention to book the flight option.' This establishes a logical sequence with search_flights. However, it doesn't explicitly mention when NOT to use it or name specific alternatives among the siblings, such as clarifying that get_flight_option_details is for information only, not booking.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_optionsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does well by explicitly stating this is a 'very cheap operation' with permission for frequent calls, and it discloses the paginated nature of the return. It also mentions filtering and sorting capabilities. However, it doesn't cover potential error conditions, rate limits beyond the 'cheap' characterization, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with three sentences that each earn their place: purpose statement, filtering/sorting capabilities, and important behavioral guidance about cost. It's front-loaded with the core purpose and appropriately sized without wasted words. The IMPORTANT section effectively highlights critical usage information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no annotations, and no output schema, the description does reasonably well. It covers the core functionality, behavioral characteristics (cheap operation, pagination), and provides usage context. However, it doesn't describe the return format beyond 'paginated list of flight options,' and with 50% schema coverage, some parameter semantics remain unclear. Given the complexity, it's mostly complete but has gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 50% schema description coverage, the description adds some value by mentioning filtering by 'price, departure and arrival times, and airlines' and 'sorting option.' However, it doesn't fully compensate for the schema coverage gap - it doesn't explain the 'search_id' parameter's purpose or format, nor does it detail the pagination parameters beyond mentioning 'paginated list.' The description provides context but leaves significant parameter semantics to the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get flight options from the previously performed search' with filtering capabilities. It distinguishes itself from 'search_flights' by focusing on retrieving and filtering results from an existing search rather than initiating a new search. However, it doesn't explicitly differentiate from 'get_flight_option_details' which presumably provides more detailed information about specific options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool: after a search has been performed ('from the previously performed search'). It implies an alternative ('search_flights' for initial searches) and includes the important guidance that this is 'very cheap operation, so you can call it as many times as needed.' However, it doesn't explicitly state when NOT to use this tool or provide direct comparisons with all sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_flightsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it performs a search, provides a search_id and description, saves options internally, requires follow-up with `get_flight_options` for detailed results, specifies time format requirements, and encourages multiple calls for optimization. It doesn't mention error handling, rate limits, or authentication needs, but covers the core operational flow well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with the core purpose, explains the process and parameters, provides important usage notes, and ends with behavioral guidance. Each sentence adds value, though the second sentence is somewhat long. The 'IMPORTANT' sections are effectively highlighted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of flight search, no annotations, and no output schema, the description does a good job of explaining the tool's operation. It covers the search process, result handling (via another tool), time format requirements, and usage patterns. It could benefit from mentioning error cases or response structure, but it provides sufficient context for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It lists the main parameters (flight segments, passengers, trip class, currency, locale) and provides context about their use in the search process. While it doesn't detail each parameter's format or constraints, it gives enough semantic understanding to guide usage, especially with the embedded schema providing detailed parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for flights using the Aviasales Flight Search API.' It specifies the verb ('search') and resource ('flights'), and mentions key parameters like flight segments, passengers, trip class, currency, and locale. However, it doesn't explicitly differentiate from sibling tools beyond mentioning that results can be retrieved with `get_flight_options`.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: it tells when to use this tool ('to find the best flight options'), mentions an alternative tool for retrieving results (`get_flight_options`), and includes important behavioral instructions ('Call this tool as many times as needed' and notes about time formats). It also implicitly guides against misuse by describing the search process.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.