Server Quality Checklist

Tool Scores

• flight_statsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool as returning aggregated stats (safe, read-only) without side effects, but does not disclose any rate limits or auth requirements. Given the simple nature, this is adequate but not exceptional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, with a front-loaded purpose sentence followed by parameter details. Every sentence adds value with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description lists the types of stats (counts, distance, unique airports/airlines, top routes). This is informative, though exact field names are not provided. The tool is relatively simple, so completeness is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains 'year' filters to a calendar year, omit for all-time, and 'upcoming_only' returns only upcoming flights. This adds meaningful context beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides aggregate stats over flights, listing specific metrics (counts, distance, unique airports/airlines, top routes). This distinguishes it from siblings get_flight (single flight) and list_my_flights (list of flights).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for aggregated statistics but does not explicitly state when to use vs alternatives. It provides filtering options (year, upcoming_only) but no when-not or alternative recommendations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_my_flightsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries transparency burden. It discloses filtering options and that output includes geo coordinates, but does not mention ordering beyond 'newest first' in limit param, pagination, error handling, or authentication needs.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is front-loaded with purpose, then parameter details. Slightly verbose with 'Args:' label, but overall efficient and clear. Could be trimmed but still good.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers input parameters well. Mentions output includes geo-coordinates for departure/arrival. With output schema present, not fully describing output is acceptable. Slight gap: doesn't explicitly state output is a list of flight legs.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description compensates fully. Each of the 5 parameters (year, after, before, upcoming_only, limit) is explained with clear semantics and usage context, e.g., 'after: Only flights departing on/after this ISO date (YYYY-MM-DD).'

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it lists the user's own flights as geo-ready legs with departure/arrival airports and coordinates. Distinguishes from siblings 'get_flight' and 'flight_stats' by focusing on list of flights with geo data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description implies use when needing a list of own flights with geo coordinates, but does not explicitly state when to use this versus siblings 'get_flight' or 'flight_stats'. No when-not or alternative guidance provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flightA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Description states it's a retrieval operation ('Get'), implying no side effects. With no annotations, it carries full burden. It specifies 'most recent' but doesn't explain chronological ordering, error handling (e.g., no matching flight), or whether the tool requires authentication. Could add more detail on behavioral guarantees.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One clear sentence plus concise parameter description. No fluff or repetition. Front-loaded with action and purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one param and output schema, description covers what input is expected and what to get back ('most recent flight leg'). Minor gaps: doesn't specify whether 'most recent' is based on departure time or schedule, or if multiple legs exist per flight.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema provides no description (0% coverage). The description adds meaning: flight_no is case- and space-insensitive. This is valuable extra info beyond schema type. Single parameter fully described.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Get'), resource ('most recent flight leg'), and criterion (matching flight number). Example 'UA194' clarifies format. Distinguishes from siblings 'flight_stats' and 'list_my_flights' by implying a single recent leg lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving the most recent leg of a specific flight, but lacks explicit guidance on when to use this tool versus alternatives like 'list_my_flights' (for listing all flights) or 'flight_stats' (for statistics). No exclusions or prerequisites stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.