Server Quality Checklist

Tool Scores

• get_flight_optionsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description mentions pagination and cheapness but omits behavioral details like error handling, response format, or side effects. For a tool with no annotations, more transparency is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences plus an important note. Purpose is front-loaded. No redundant text. Could be slightly more structured but overall concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks explanation of response format, error cases, required search_id from prior step, and how pagination works. Complex tool with nested filters needs more detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%; description vaguely lists filter criteria (price, times, airlines) but does not detail the filters object structure. Adds marginal value over schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves flight options from a prior search, with filtering and pagination. It distinguishes from siblings: get_flight_option_details (details), request_booking_link (booking), and search_flights (initial search).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states 'from the previously performed search' implying prerequisite. Notes the operation is cheap and callable many times. Does not explicitly mention when not to use, but context with siblings provides differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_option_detailsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses output contents (segments, price, baggage) but does not mention side effects, prerequisites, authentication, or error conditions. The implied dependency on previous search is not emphasized.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, no fluff. Front-loads the purpose, then lists contents. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacks output schema, but description mentions return fields. It provides enough context for a simple retrieval tool, though could clarify the connection to search_flights more explicitly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with both parameters well-described. The description adds general context about output but not parameter-specific meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'retrieve' and the resource 'detailed information about a specific flight option'. It mentions included fields (segments, price, baggage info) and distinguishes from siblings like get_flight_options (likely returns list) and request_booking_link.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'It is useful for getting more granular information' but does not explicitly state when to use or when not to use it relative to siblings. There is no mention of prerequisites or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• request_booking_linkA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It states that the tool generates a booking link, indicating a side effect of creating an external resource. However, it lacks details on required permissions, rate limits, or what the output link contains.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, starting with the core purpose. It is efficient and avoids redundancy, though the two sentences could be merged without loss of clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description should clarify the return value. It mentions generating a booking link but does not specify whether the link is returned in the response or how it is structured. This leaves ambiguity about what the agent will receive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already provides thorough descriptions for all three parameters (100% coverage). The description adds no additional parameter-level information beyond what the schema offers, resulting in a baseline score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('request link') and resource ('booking a flight option'). It distinguishes from sibling tools like search_flights and get_flight_option_details by focusing on generating a booking link.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends using this tool after the user expresses intention to book, providing clear context for when to invoke it. However, it does not mention when not to use it or list alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_flightsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that search results are saved internally, that all times are local and in HH:MM 24-hour format, but does not mention auth or rate limits. It is fairly transparent, but could include more about statefulness.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections and front-loaded purpose. It is slightly verbose but each sentence provides value, earning a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of the nested schema and the lack of an output schema, the description adequately explains the workflow (search then retrieve options) and provides critical usage details. It is practically complete for an agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema_description_coverage is 0% (likely a metric artifact), the input schema has detailed descriptions for all nested properties. The tool description adds value with examples and important usage hints, going beyond what the schema provides. Baseline 3 is exceeded.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it searches for flights using Aviasales API, provides a search_id, and saves options internally. It explicitly mentions the subsequent use of get_flight_options, distinguishing itself from its siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance on when to call this tool (as many times as needed) and when to use get_flight_options. It includes examples for one-way, round trip, multi-city, and warns against using multiple segments for alternative dates.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.