Server Quality Checklist

Tool Scores

• lookup_airportA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the burden of behavioral disclosure. It discloses the tool's ability to handle partial names and return metro codes, which is useful. However, it does not explain what the tool returns (e.g., multiple matches, structure), how it handles ambiguity or no results, or any other edge cases. For a lookup tool, this leaves some uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core purpose in the first sentence. It is concise, contains no filler, and every sentence adds useful context: the first defines the function, the second provides usage guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 parameters, no output schema), and the description provides purpose and usage, but it omits explanation of the 'limit' parameter and does not mention what the response looks like. Given no annotations and no output schema, the description is somewhat incomplete for an agent to fully anticipate the tool's behavior, but it covers the primary intent adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes the 'query' parameter with examples, and the tool description reinforces its purpose by mentioning partial names and metro codes. However, the 'limit' parameter is not mentioned in the description at all, and its schema lacks a description. With 50% schema coverage, the description does not compensate for the missing limit semantics, leaving the agent to infer its role from the name and constraints alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: resolving partial city or airport names to IATA codes. It uses the specific verb 'resolve' and names the resource ('partial city or airport name to IATA codes'). This sets it apart from sibling tools like search_flights or get_itinerary_details, which deal with routing and itineraries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use this tool: 'Use this when the user names a place rather than a code, or to find the metro code that covers every airport in a city.' This gives clear context and effectively distinguishes it from alternatives. It also implies that for other operations (e.g., flight search), other tools should be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_itinerary_detailsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses the read-only nature implicitly by focusing on 'show' details, specifies the output granularity (per-segment) and exact data points, and notes session reuse and performance ('fast'). Missing are error conditions, but for a detail-retrieval tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact sentences: the first states the purpose and output fields, the second adds behavioral context about session reuse and speed. Every word earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter tool with no output schema, the description covers the essential context: what it does, what it returns, when it is used, and a performance note. It implicitly states the prerequisite (a prior search) and the field list serves as the return-value specification. No critical gaps for this complexity level.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters (searchId, rank) already described in the schema. The description adds context that the itinerary is from a previous search, but no new parameter-specific semantics beyond the schema's explicit examples and definitions. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Show') and resource ('per-segment detail for one itinerary from a previous search') with specific fields (operating flight numbers, booking class, fare basis codes, aircraft type). This differentiates it from siblings like search_flights, which perform the initial search, and lookup_airport, which handles airport lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates the tool is for expanding a previously returned itinerary ('from a previous search', 'Reuses the earlier search session'), providing clear context for when to use it. It does not explicitly name an alternative or state when not to use it, but the session-reuse detail effectively excludes standalone use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• routing_language_referenceA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining what the tool returns—a comprehensive syntax reference. It discloses the scope of content covered, which is the key behavioral trait for a documentation tool. It doesn't detail the output format, but the nature of a reference makes that less critical.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with a front-loaded title and a concise list of topics. Every word earns its place, and the 'Read this before...' directive adds practical utility without bloat.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description sufficiently conveys the tool's purpose and scope for a simple zero-parameter reference. It doesn't enumerate every syntax detail, but that's expected—the tool itself provides the full syntax. It's complete enough to guide invocations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there are no parameter semantics to clarify. The description instead explains what the reference covers, which is useful context beyond the empty schema. Baseline 4 is appropriate for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing the full syntax for the 'routing' and 'extraCodes' fields, listing specific topics like carrier pinning, alliances, and exclusions. This distinguishes it from sibling search tools, establishing it as a reference/documentation resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises reading this before composing a non-trivial routing code, which is clear when-to-use guidance. It doesn't explicitly exclude other scenarios or name alternative tools, but the context of siblings makes its purpose unambiguous.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_flexible_datesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior, and it does: 'Runs one search per date, so keep ranges modest (a 14-day scan takes several minutes).' This reveals a key performance characteristic. However, it does not describe the output format or any other behavioral details, though as a search tool there are few side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core action, then a performance caveat, then an explicit alternative. Every word contributes useful information with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (17 parameters, no output schema), the description gives a clear high-level purpose, a performance warning, and a usage boundary. It could describe the return format or key required parameters, but the core usage is well covered for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 53%, so the schema already explains many parameters. The description adds context about range size and performance, which indirectly relates to startDate/endDate and maxDates, but it does not explain any specific parameter in detail beyond what the schema provides. It does not compensate significantly for the undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Scan a range of departure dates and report the cheapest fare on each,' using specific verbs and a resource. It also distinguishes from the sibling search_flights by explicitly directing users with single-date flexibility to that alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance is provided: 'For a single date with ±flexibility, use search_flights with flexDays instead.' This tells the agent when not to use this tool and names the alternative. It also advises keeping ranges modest due to the one-search-per-date behavior, giving practical usage constraints.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_flightsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses latency ('Expect 20-60s'), return value ('ranked itineraries and a Search ID'), and slice semantics. It does not cover error cases or rate limits, but the disclosed behaviors are meaningful for invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: two sentences plus a bulleted list convey the core concept, routing flexibility, latency, and follow-up flow. Every sentence earns its place, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex (13 params, multi-slice logic) and has no output schema. The description covers the essential mental model, routing mechanism, latency, and connection to get_itinerary_details. It does not enumerate return field details or error handling, but for a search tool paired with a details tool, it provides sufficient context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 54%, so the description adds valuable extra meaning. It explains the slices parameter's semantics (one-way/round-trip/open-jaw) and the routing parameter's purpose (pin carrier, force connection, control flight count). Other parameters like passenger counts remain undocumented in both schema and description, but the most complex parameters receive useful added context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Search ITA Matrix for itineraries' – a specific verb+resource pair. It then distinguishes trip shapes (one-way, round-trip, multi-city) and explicitly mentions the Search ID for get_itinerary_details, clearly separating this tool from the details tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: slice count determines trip type, routing codes allow complex legs, and results feed into get_itinerary_details. It does not explicitly contrast with search_flexible_dates or state when not to use this tool, so it stops short of full alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.