Server Quality Checklist

Tool Scores

• fare_product_matchA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden for behavioral disclosure. It does not detail side effects, authentication needs, rate limits, or what happens with invalid inputs. The description only covers purpose, not runtime behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with a clear list of outputs and criteria. It is front-loaded with the main action and contains no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters, no output schema, and no annotations. The description explains the purpose and high-level logic but does not describe return format or success/failure conditions. For a recommendation tool, an agent might need to know what the output looks like (e.g., JSON structure with fare product type). Thus, completeness is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but the description adds meaning beyond parameter names by linking them to fare product logic (e.g., 'direction' and 'backtracking' correspond to isOneDirection and includeBacktracking). This helps an agent understand how parameters influence the recommendation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: recommend the best fare product type for a route, listing specific examples (RTW, Circle Pacific, Circle Atlantic, Open Jaw, Custom Multi-City). It also mentions key criteria considered (stop count, direction, backtracking), making it distinguishable from sibling tools like custom_route_build or plan_route.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives. It implies usage for fare product recommendation but does not state when not to use it or mention prerequisites. Sibling tools like route_validate or hub_check serve different purposes, but no comparison is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• route_suggestA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully disclose behavior. It mentions returns templates with ratings, suggesting read-only operation, but does not state mutability, authentication needs, or error handling. Moderately transparent but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences. The most critical information (what the tool does and its output) is front-loaded. No redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and no annotations, the description lacks details on return format, error scenarios, and edge cases. It says 'returns up to 3 templates' but does not define what a 'template' contains, leaving potential ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description repeats parameter concepts (regions, direction, alliance) without adding new semantics beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it suggests multi-stop flight routings based on regions, direction, and alliance, which differentiates it from siblings like custom_route_build or plan_route. However, it could more explicitly contrast with these alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for trip planning inspiration but does not specify when not to use it or mention alternative tools for other tasks. No explicit guidance on prerequisites or context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• custom_route_buildA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It discloses the output (individually-ticketable segments with carrier recommendations) and scenarios handled (mixed carriers, LCCs, Gulf bridge connections, surface sectors). However, it does not mention side effects, auth needs, or statefulness, which would enhance transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each earning its place: core purpose, handling details, credibility statement, usage guidance. No fluff, front-loaded with key info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given complexity (4 params, no output schema, no annotations), the description explains the tool's role and niche well. It lacks details on return format or what 'carrier recommendations' entail, but is sufficient for an agent to understand when and why to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context about why you'd use the tool (complex itineraries) but does not elaborate on individual parameters beyond what schema already provides. It helps understand overall purpose but not per-parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool breaks complex multi-city itineraries into individually-ticketable segments with carrier recommendations. It specifies the verb 'break' and resource 'multi-city itinerary', and distinguishes from sibling tools by mentioning handling routes that don't fit alliance fare rules, mixed carriers, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this for any route with 4+ stops, backtracking, or region combinations that alliance fares can't cover,' giving clear when-to-use guidance. It implies when not to use (simpler routes) but does not name specific alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• hub_checkA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the tool's behavior: checks connections, identifies dead legs, suggests fixes, and shows carrier combos. It does not mention permissions or side effects, but the behavior is sufficiently transparent for a read-like analysis tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with three sentences: first states purpose, then lists capabilities, then gives use cases. It is front-loaded and contains no redundant or irrelevant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity and absence of output schema, the description covers purpose and usage well. It explains what the tool does and when to use it, but lacks explicit information about output format or return values, which could be inferred from the described behaviors.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 100% coverage with descriptions for each parameter. The description adds context about alliance fares but does not elaborate on parameters beyond what the schema provides. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks the best connection between two airports and lists specific capabilities like identifying dead legs and suggesting hub routing fixes. It also distinguishes from siblings by focusing on hub routing and alliance fares, making it essential for specific routing contexts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool, citing transpacific, kangaroo, and intra-Asia routing. However, it does not explicitly state when not to use it or compare with sibling tools, missing an opportunity to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• route_validateA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses core behaviors: checking alliance rules, dead legs, poison carriers, and estimating bookability. This provides insight beyond the schema, though it doesn't explicitly state read-only nature or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence states the purpose, the second gives usage guidance. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with full schema coverage and no output schema, the description explains the tool's functionality well. However, it omits details about the return format (e.g., a boolean or list of issues), which could be important for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds context about ordered IATA codes and optional per-leg carriers, but the schema already describes parameters well. The description doesn't significantly enhance parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool validates multi-city flight routing feasibility, checking alliance rules, dead legs, poison carriers, and bookability. This distinguishes it from sibling tools like custom_route_build or fare_product_match.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly advises using this tool before building an itinerary to catch problems early, giving clear context. It doesn't list specific alternatives or when not to use, but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• trip_idea_createA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. It discloses that it automatically runs plan_route and that the consultant starts informed. Does not mention permissions or side effects, but for a creation tool, adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with main action, efficient and no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, process, and usage context. Lacks info on return value or system response, but acceptable given the tool's nature and no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline is 3. Description adds no extra insight beyond schema; mentions automatic plan_route but does not elaborate on parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the action ('Create a trip idea') and the outcome ('hands off to a human consultant'). Distinguishes from sibling tools by mentioning it automatically runs plan_route.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this when the customer is ready to get a real quote.' Implies that for routing analysis alone, plan_route might be used separately, but no explicit alternatives listed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• plan_routeA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool automatically evaluates three types of builds, handles direction detection, backtracking, and carrier selection, which provides sufficient behavioral insight for a computational tool. It does not discuss failure modes or concurrency, but is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four sentences long, front-loaded with the primary purpose ('primary entry point for any multi-city trip'). Each sentence adds meaningful information without being verbose, though could be slightly tighter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately covers the tool's role as a primary analysis tool. It explains inputs and internal logic sufficiently for an agent to decide when to invoke it. Lacks return value description but is acceptable for this type of tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% (all parameters have descriptions), baseline is 3. The description adds value beyond the schema by explaining that 'budget affects carrier selection' and 'cities are ordered list', and that preferences are travel preferences, which clarifies usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly defines this as the primary entry point for multi-city trips, automatically evaluating different alliance builds and recommending the best approach. It distinguishes itself from sibling tools like custom_route_build (which likely requires manual input) and route_suggest.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states it is the 'primary entry point', implying it should be used first for multi-city trips. It outlines what the tool handles (direction detection, backtracking, alliance feasibility, etc.), but does not explicitly state when to use alternatives or provide exclusions, though the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.