Server Quality Checklist

Tool Scores

• search_flightsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action ('search') without any details about permissions, rate limits, pagination, or what the search returns (e.g., flight details, statuses). For a search tool with no annotations, this is insufficient to inform the agent about operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It is appropriately sized for a simple search tool and front-loads the core action ('search for flights'), making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what the search returns (e.g., flight objects, statuses), how results are ordered, or any limitations (e.g., real-time vs. historical data). Without annotations or output schema, the description should provide more context to compensate, but it fails to do so.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are well-documented in the schema itself (e.g., IATA/ICAO codes, flight status with enum, limit with default and max). The description adds no additional parameter semantics beyond 'by various criteria', which is redundant. Baseline is 3 when schema coverage is high, as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the purpose as 'Search for flights by various criteria', which is clear but vague. It specifies the verb 'search' and resource 'flights', but lacks specificity about what kind of search or how it differs from sibling tools like 'get_flight_data' or 'get_flight_status'. It doesn't provide enough detail to distinguish it from alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools 'get_flight_data' or 'get_flight_status'. It mentions 'by various criteria', which implies flexibility, but offers no explicit when-to-use or when-not-to-use instructions, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_dataB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool 'Get[s] real-time data,' implying a read operation, but doesn't specify if it requires authentication, has rate limits, returns structured data, or handles errors. For a tool with no annotation coverage, this leaves significant gaps in understanding its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It is front-loaded with the core action and resource, making it easy to parse quickly. Every part of the sentence contributes essential information, earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is adequate but incomplete. It covers the basic purpose but lacks details on usage guidelines, behavioral traits, and how it fits with siblings. Without an output schema, it doesn't explain return values, which is a gap, but the concise purpose statement provides a minimal viable foundation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal meaning beyond the input schema, which has 100% coverage. It implies the tool uses a flight number parameter, but doesn't clarify the relationship between 'flight_iata' and 'flight_icao' or why both are options. Since the schema already documents these parameters well, the baseline score of 3 is appropriate, as the description doesn't significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get real-time data for a specific flight by flight number.' It specifies the verb ('Get'), resource ('real-time data'), and scope ('specific flight by flight number'), making the intent unambiguous. However, it doesn't explicitly differentiate from sibling tools like 'get_flight_status' or 'search_flights', which might offer overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions retrieving data 'by flight number,' but doesn't clarify if this is for a single flight lookup, how it differs from 'get_flight_status' (which might focus on status updates) or 'search_flights' (which could handle broader queries). No exclusions, prerequisites, or context for tool selection are included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states the basic action without disclosing behavioral traits like data freshness (e.g., real-time vs. cached), error handling (e.g., invalid flight codes), or rate limits. It's minimal and lacks critical operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste, clearly front-loading the core purpose. It's appropriately sized for a simple lookup tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (simple query with two parameters) and full schema coverage, the description is adequate but incomplete due to missing output details (no output schema) and insufficient behavioral context for a tool with no annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the two parameters (flight_iata, flight_icao) and their oneOf logic. The description adds no additional meaning beyond implying flight number usage, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('current status of a flight'), making the purpose specific and actionable. It distinguishes from 'get_flight_data' and 'search_flights' by focusing on status retrieval, though it doesn't explicitly differentiate them in the text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings ('get_flight_data', 'search_flights'), such as for real-time status vs. historical data or search queries. It also lacks prerequisites like flight number availability.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.