Server Quality Checklist

Tool Scores

• get_airport_info_fullC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a read operation ('Returns'), which is clear, but lacks details about rate limits, error handling, authentication needs, or response format. The description doesn't explain what happens if the code is invalid or if data is unavailable, leaving behavioral gaps for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences: one listing the returned information and another stating the requirement. It's front-loaded with the core purpose. However, the list of fields could be slightly condensed (e.g., 'localization' might be vague), and it lacks structural markers like bullet points, but overall it's efficient with zero wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and high schema coverage, the description is minimally adequate. However, with no annotations and no output schema, it fails to explain the return format or behavioral traits like error handling. For a read-only tool, this is a moderate gap, as agents need to understand what 'detailed airport information' entails in practice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'code' fully documented in the schema as 'Airport IATA or ICAO code.' The description adds minimal value beyond this, only reiterating that the code is 'REQUIRED' and 'must be provided and non-empty,' which is already implied by the schema's required field and minLength constraint. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns detailed airport information' with specific fields listed (full name, codes, localization, etc.). It distinguishes from the sibling 'get_airport_info_light' by emphasizing 'detailed' and listing comprehensive fields, though it doesn't explicitly contrast with that sibling. The verb 'Returns' is specific and the resource 'airport information' is well-defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions a required parameter but offers no context about when this detailed airport info is needed over the 'light' version or other flight-related tools. There's no mention of prerequisites, alternatives, or exclusions, leaving usage decisions unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_airport_info_lightC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states the tool returns data and has a required parameter, but doesn't mention error handling, rate limits, authentication needs, or what happens with invalid codes. For a read operation with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that each serve a purpose: the first states what the tool returns, the second specifies a requirement. There's no unnecessary information, though it could be more front-loaded with usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that returns data. It doesn't explain the format or structure of returned information (e.g., whether it's a single object or list), error responses, or how it differs from the 'full' version, leaving significant gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter. The description adds that the code 'must be provided and non-empty,' which slightly reinforces the schema's required constraint and minLength, but doesn't provide additional semantic meaning beyond what's in the structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns airport name, ICAO and IATA codes.' It specifies the verb ('Returns') and resource ('airport name, ICAO and IATA codes'), but doesn't explicitly differentiate from sibling tools like 'get_airport_info_full' beyond the 'light' naming convention.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions a required parameter constraint but doesn't explain when to choose this 'light' version over 'get_airport_info_full' or other airport-related tools, leaving usage context unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_tracksC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool returns positional tracks, implying a read-only operation, but doesn't cover critical aspects such as whether it requires authentication, rate limits, data freshness (live vs. historic), error handling, or the format of the returned tracks. This leaves significant gaps in understanding the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise and front-loaded, consisting of two sentences that directly state the tool's purpose and a key requirement. There is no wasted language or redundancy, making it efficient for quick comprehension by an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of flight tracking and the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'positional tracks' entail (e.g., coordinates, timestamps, altitude), how data is sourced, or potential limitations. For a tool with no structured behavioral or output information, this leaves the agent poorly equipped to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal value beyond the input schema. It emphasizes that 'flight_id must be provided and non-empty,' which reinforces the schema's 'required' and 'minLength' constraints but doesn't provide additional semantic context. With 100% schema description coverage, the baseline is 3, as the schema already documents the parameter adequately, and the description doesn't compensate with extra insights like examples or usage tips.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns positional tracks of a specific flight.' It specifies the verb ('Returns') and resource ('positional tracks of a specific flight'), making it easy to understand what the tool does. However, it doesn't explicitly differentiate from siblings like 'get_historic_flights_positions_full' or 'get_live_flights_positions_full', which might also return flight positions but with different scopes or details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions that 'flight_id must be provided and non-empty,' but this is a requirement rather than usage context. There's no indication of when this tool is appropriate compared to sibling tools like 'get_flight_summary_full' or position-related tools, leaving the agent without clear selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historic_flights_positions_lightB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the data source (FR24 API) and historical scope (back to May 11, 2016, depending on subscription), which adds some context. However, it lacks critical behavioral details such as rate limits, authentication needs, error handling, pagination, or what the output looks like (e.g., format, structure). For a tool with 17 parameters and no annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first, followed by API context and an important usage note. It uses three sentences efficiently, with no redundant information. However, it could be slightly more structured by separating usage guidelines into distinct points for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (17 parameters, no annotations, no output schema), the description is incomplete. It covers the basic purpose and some parameter constraints but lacks essential context such as output format, error conditions, rate limits, and detailed behavioral traits. For a historical data query tool with many search options, more comprehensive guidance is needed to ensure the agent can use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are documented in the input schema itself. The description adds minimal parameter semantics beyond the schema, only emphasizing that 'timestamp is required, and at least one additional search parameter (other than limit) must be provided and non-empty.' This provides some constraint context but doesn't add substantial meaning over the schema's detailed descriptions. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns historical aircraft flight movement information including latitude, longitude, speed and altitude.' It specifies the verb ('returns'), resource ('historical aircraft flight movement information'), and data fields. However, it doesn't explicitly differentiate from sibling tools like 'get_historic_flights_positions_full' or 'get_historic_flights_count,' which would be needed for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage context by mentioning the FR24 API's historical data availability and subscription dependencies, and it includes an 'IMPORTANT' note about required parameters. However, it doesn't explicitly state when to use this tool versus alternatives like the 'full' version or other historical flight tools, nor does it provide clear exclusions or prerequisites beyond the parameter requirements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_summary_countB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions that certain parameters are 'required' and provides a constraint ('at least one additional search parameter'), which adds some behavioral context. However, it doesn't describe other important traits like whether this is a read-only operation (implied by 'returns'), potential rate limits, authentication needs, error handling, or the format of the returned count (e.g., integer, JSON structure). For a tool with no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, with the core purpose stated first ('Returns the number of flights for a given flight summary query.') followed by an important usage note. It consists of two sentences that earn their place by clarifying requirements, with no redundant or vague language. However, it could be slightly improved by integrating the purpose and constraints more seamlessly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (10 parameters, no output schema, no annotations), the description is moderately complete. It covers the purpose and basic usage constraints, but lacks details on behavioral aspects (e.g., safety, performance) and output format. Since there's no output schema, the description should ideally hint at what 'number of flights' means (e.g., a simple integer count or structured response), but it doesn't, leaving gaps for an agent to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, meaning all parameters are well-documented in the input schema itself (e.g., formats like 'YYYY-MM-DDTHH:MM:SSZ', constraints like 'max 15'). The description adds minimal value beyond the schema by emphasizing that the datetime parameters are required and that at least one additional parameter is needed, but it doesn't provide new semantic details about the parameters. Given the high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns the number of flights for a given flight summary query.' It specifies the verb ('returns'), resource ('number of flights'), and scope ('flight summary query'). However, it doesn't explicitly differentiate from sibling tools like 'get_historic_flights_count' or 'get_live_flights_count', which also return flight counts but for different contexts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage guidance by stating that 'flight_datetime_from and flight_datetime_to are required, and at least one additional search parameter should be provided.' This implies when to use it (for flight summary queries with datetime constraints). However, it doesn't explicitly mention when to use this tool versus alternatives like 'get_flight_summary_full' (which likely returns detailed data) or other count tools for historic/live flights, leaving the context somewhat implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_airline_infoB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the required ICAO code constraint, which is useful, but lacks details on other behavioral traits like error handling, rate limits, authentication needs, or what happens if the ICAO code is invalid. For a read-only tool with no annotations, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey the tool's purpose and key requirement. Every sentence earns its place: the first states what it returns, and the second specifies the mandatory input, with no wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (single parameter, no output schema, no annotations), the description is adequate but has clear gaps. It covers the basic purpose and input requirement, but without annotations or output schema, it should ideally include more about return values or error cases to be fully complete for agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'icao' parameter documented as 'Airline ICAO code.' The description adds value by emphasizing that 'icao code must be provided and non-empty,' reinforcing the requirement beyond the schema's 'minLength: 1' and 'required' field. However, it doesn't provide additional semantic context, such as format examples or validation rules, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns airline name, ICAO and IATA codes.' It specifies the verb ('Returns') and resource ('airline name, ICAO and IATA codes'), making the function unambiguous. However, it doesn't explicitly differentiate from sibling tools like get_airport_info_full or get_flight_summary_count, which focus on airports and flights rather than airlines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance by stating 'REQUIRED: icao code must be provided and non-empty,' which indicates when to use this tool (when you have an ICAO code). However, it doesn't explicitly mention when not to use it or suggest alternatives among the sibling tools, such as using get_airport_info_full for airport data instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_summary_lightA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions data availability starting from 2024-04-07 and future extensions, which adds useful context. However, it lacks details on rate limits, authentication needs, error handling, or pagination behavior, leaving gaps for a tool with 12 parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by data availability details and important usage notes. It's concise with three sentences that each serve a clear purpose, though it could be slightly more streamlined by integrating the date requirement into the initial sentence.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (12 parameters, no output schema, no annotations), the description is moderately complete. It covers the purpose, data scope, and key usage rules, but lacks details on output format, error cases, or performance considerations, which would be helpful for a tool with many search options.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds value by emphasizing that 'flight_datetime_from and flight_datetime_to are required, and at least one additional search parameter (other than sort and limit) should be provided,' which clarifies usage constraints beyond the schema's required fields.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns 'key timings and locations of aircraft takeoffs and landings alongside all primary flight, aircraft, and operator information,' which is specific about the data returned. However, it doesn't explicitly differentiate this 'light' version from sibling tools like 'get_flight_summary_full' or 'get_flight_summary_count,' leaving some ambiguity about scope differences.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage by stating 'Both real-time and historical data are available' and specifying date range requirements and additional parameter needs. It doesn't explicitly mention when to use this tool versus alternatives like 'get_flight_summary_full' or 'get_flight_summary_count,' but the naming and data scope imply it's a summary-focused tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historic_flights_countA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that timestamp is required and at least one additional parameter is needed, which are important behavioral constraints. However, it doesn't mention rate limits, authentication requirements, data freshness, or what happens when no results match the criteria. For a tool with 17 parameters and no annotations, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and well-structured. The first sentence states the purpose, and the second sentence provides critical usage constraints. Every word earns its place, with no redundant information. It's front-loaded with the core functionality followed by essential requirements.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (17 parameters, no annotations, no output schema), the description is incomplete. While it covers the basic purpose and parameter requirements, it doesn't explain what the return value looks like (just 'number' without format), error conditions, or how the count relates to the filtering parameters. For a count tool with rich filtering options, more context about the counting methodology would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 17 parameters thoroughly. The description adds minimal value beyond the schema by emphasizing that timestamp is required and at least one other parameter (excluding limit) must be provided. This provides some context about parameter dependencies but doesn't add semantic meaning to individual parameters beyond what's in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns number of historical aircraft flight positions.' It specifies the resource (historical aircraft flight positions) and verb (returns number). However, it doesn't explicitly differentiate from sibling tools like 'get_historic_flights_positions_full' or 'get_flight_summary_count', which might offer similar functionality with different output formats.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage guidance: 'Timestamp is required, and at least one additional search parameter (other than limit) must be provided and non-empty.' This tells the agent when to use this tool (for historical counts with timestamp and at least one filter). However, it doesn't specify when to choose this tool over alternatives like 'get_historic_flight_events_full' or 'get_live_flights_count'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flight_summary_fullA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It does well by specifying data availability timeframe (starting 2024-04-07), indicating future extensions, and stating important parameter constraints. However, it doesn't mention rate limits, authentication requirements, pagination behavior, or response format details that would be helpful for a complex query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with three sentences that each earn their place: first states core functionality, second adds data availability context, third provides critical parameter guidance. It's front-loaded with the main purpose and avoids unnecessary elaboration. Could be slightly more concise by combining some information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 12-parameter query tool with no annotations and no output schema, the description provides adequate but incomplete context. It covers the core purpose, data availability, and parameter requirements, but lacks information about response format, pagination, rate limits, error conditions, or how results differ from sibling tools. The absence of output schema increases the need for more behavioral detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 12 parameters thoroughly. The description adds minimal parameter-specific information beyond what's in the schema - it only emphasizes that flight_datetime_from/to are required and at least one additional search parameter should be provided. This meets the baseline 3 when schema does heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns comprehensive timings and locations of aircraft takeoffs/landings with detailed flight, aircraft, and operator information. It specifies both real-time and historical data availability. However, it doesn't explicitly differentiate from siblings like get_flight_summary_light or get_flight_summary_count, which likely provide less comprehensive data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context: it states both real-time and historical data are available, specifies data availability starting from 2024-04-07, and gives important parameter requirements (flight_datetime_from/to required, plus at least one additional search parameter). It doesn't explicitly say when to use this vs. sibling tools like get_flight_summary_light, but the 'comprehensive' vs 'light' naming implies this is the full-featured version.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historic_flight_events_lightA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It describes the return format (sorted by event_timestamp, grouped by flight_id) and lists available event types, which adds value beyond the input schema. However, it doesn't disclose important behavioral traits like rate limits, authentication requirements, error handling, or what happens when invalid parameters are provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with two sentences that each serve a clear purpose: the first describes what the tool returns, the second states requirements. It's front-loaded with the core functionality. However, the list of event types in parentheses is somewhat verbose and could be more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only query tool with 2 parameters and 100% schema coverage but no output schema, the description provides adequate information about what data is returned and parameter requirements. However, without annotations or output schema, it lacks details about response format, pagination, error conditions, and performance characteristics that would be helpful for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds minimal value beyond the schema by restating that both parameters are required and non-empty, and listing the available event types (which is also in the schema). This meets the baseline expectation when schema coverage is high.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Returns') and resource ('selected historical flight events'), lists specific event types, and distinguishes from sibling tools by specifying it's a 'light' version focused on events rather than positions, summaries, or counts. It provides more specificity than just restating the name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about what data is returned (events sorted by timestamp and grouped by flight_id) and includes a REQUIRED section indicating flight_ids and event_types must be provided and non-empty. However, it doesn't explicitly state when to use this tool versus alternatives like get_historic_flight_events_full or other sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historic_flights_positions_fullA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool returns data (read operation) and mentions subscription plan dependencies, which is useful context. However, it doesn't describe important behavioral aspects like rate limits, authentication requirements, error conditions, pagination, or what happens when no results are found. For a complex 17-parameter tool with no annotations, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with three sentences that each serve a purpose: first defines what the tool returns, second provides API context and subscription details, third states critical requirements. It's front-loaded with the core functionality. Could be slightly more concise by combining some information, but overall well-structured with zero wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (17 parameters, no annotations, no output schema), the description provides adequate basic information about what the tool does and its requirements. However, for such a complex query tool with many filtering options and no output schema, the description should ideally provide more guidance about result format, typical use cases, or limitations. The absence of annotations means the description should do more heavy lifting than it currently does.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 17 parameters thoroughly. The description adds minimal parameter semantics beyond the schema - it only emphasizes that 'timestamp is required' and 'at least one additional search parameter (other than limit) must be provided'. This meets the baseline for high schema coverage but doesn't add significant value beyond what's already in the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'returns historical aircraft flight movement information' with specific details like 'latitude, longitude, speed, and altitude' and 'key flight and aircraft information such as origin, destination, callsign, registration and aircraft type'. It distinguishes from sibling tools by specifying it's for historical data (vs. live flights) and full positions (vs. light versions or counts).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: for historical flight data 'dating back to May 11, 2016' with subscription dependencies. It explicitly states requirements: 'Timestamp is required, and at least one additional search parameter (other than limit) must be provided and non-empty.' However, it doesn't explicitly mention when NOT to use it or name specific alternatives among the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historic_flight_events_fullA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses that the tool returns sorted and grouped data, which is valuable behavioral context. However, it doesn't mention rate limits, authentication needs, error conditions, pagination, or what constitutes 'detailed information' beyond the event types listed. The description adds some behavioral insight but leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the purpose and key characteristics, the second specifies requirements. Every element serves a clear purpose with no redundant information. The event type list is appropriately included for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only query tool with 2 parameters and 100% schema coverage, the description provides adequate purpose and requirements. However, without annotations or output schema, it should ideally describe the return format more explicitly (what 'detailed information' includes) and mention any constraints like rate limits. The sibling context suggests this is part of a flight data API where such details matter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents both parameters. The description adds the requirement that parameters must be 'non-empty' and implies they work together, but doesn't provide additional semantic context beyond what's in the schema descriptions. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Returns selected historical flight events'), identifies the resource ('historical flight events'), and lists the specific event types included. It distinguishes itself from sibling 'get_historic_flight_events_light' by specifying 'with detailed information' and the grouping/sorting approach.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (for historical flight events with detailed information), and the REQUIRED section indicates mandatory parameters. However, it doesn't explicitly contrast when to use this versus the 'light' version or other flight-related tools, nor does it mention any exclusions or prerequisites beyond parameter requirements.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_live_flights_positions_lightA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It effectively discloses key behavioral traits: it's a read operation (implied by 'Returns'), specifies the real-time nature of data, and states the mandatory parameter requirement. However, it doesn't mention rate limits, authentication needs, or data freshness details that would be helpful for a real-time API.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with purpose first, then critical constraint, then parameter list. Every sentence earns its place, though the long parameter list could be slightly more organized. It's appropriately sized for a tool with 16 parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (16 parameters, real-time data, no output schema), the description is reasonably complete. It covers purpose, constraints, and parameter scope well. However, without annotations or output schema, it could benefit from mentioning response format or data limitations. The 100% schema coverage helps compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 16 parameters thoroughly. The description adds value by listing all parameter names and emphasizing the 'at least one required' constraint, but doesn't provide additional semantic context beyond what's in the schema descriptions. Baseline 3 is appropriate when schema does heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Returns real-time aircraft flight movement information') and resources ('aircraft flight'), and distinguishes it from siblings by specifying it's for 'real-time' positions (vs. historic positions in sibling tools like get_historic_flights_positions_full). The inclusion of specific data fields (latitude, longitude, speed, altitude) further clarifies scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance with 'IMPORTANT: At least one search parameter (other than limit) must be provided and non-empty' and lists all valid search parameter options. It distinguishes when to use this tool (for real-time positions) versus siblings (for historic data, counts, summaries, etc.), though it doesn't explicitly name alternatives, the context makes alternatives clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_live_flights_countA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively communicates the constraint that at least one non-limit parameter is required, which is crucial behavioral information not evident from the schema alone. However, it doesn't mention rate limits, authentication requirements, or what happens when no matches are found.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured: a clear purpose statement followed by an important constraint, then a comprehensive parameter list. Every sentence earns its place, with no redundant information. The front-loaded constraint is particularly effective for agent decision-making.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a query tool with 16 parameters and no output schema, the description provides good context about the mandatory parameter requirement and available search dimensions. However, without annotations or output schema, it could benefit from mentioning what the count output looks like (e.g., integer, format) or any performance considerations for this real-time data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 16 parameters thoroughly. The description adds minimal value beyond listing parameter names - it doesn't provide additional context about parameter interactions, precedence, or usage patterns beyond the schema's individual parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Returns') and resource ('count of real-time aircraft flights'), and distinguishes it from siblings by specifying it's about 'count' rather than detailed positions or summaries. It explicitly mentions 'real-time' which differentiates it from historic flight tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'IMPORTANT: At least one search parameter (other than limit) must be provided and non-empty.' It also lists all available search parameters, giving clear context for when to use this tool versus alternatives that might focus on different data types or detail levels.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_live_flights_positions_fullA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it returns real-time data (implying dynamic, current information) and lists the types of information included. However, it doesn't mention potential limitations like rate limits, data freshness, or error conditions (e.g., what happens if no flights match the filters). For a real-time data tool with no annotations, this is good but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured: the first sentence clearly states the purpose and returned data, followed immediately by the critical usage requirement. Every sentence earns its place—the first explains what the tool does, and the second provides essential invocation guidance. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (16 parameters, real-time data, no annotations, no output schema), the description does well. It clearly explains the tool's purpose, provides crucial usage rules, and lists available parameters. However, it doesn't describe the output format (e.g., structure of returned flight data) or potential behavioral aspects like pagination or error handling, which would be helpful given the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 16 parameters thoroughly. The description adds value by emphasizing that at least one parameter (other than limit) must be provided, which is a critical semantic constraint not captured in the schema's property definitions. However, it doesn't provide additional context about parameter interactions or prioritization beyond what's in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns real-time aircraft flight movement information including latitude, longitude, speed, and altitude alongside key flight and aircraft information such as origin, destination, callsign, registration and aircraft type.' This specifies both the data returned (positional data plus flight metadata) and distinguishes it from sibling tools like 'get_live_flights_positions_light' (likely a lighter version) and 'get_historic_flights_positions_full' (historical vs real-time).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'IMPORTANT: At least one search parameter (other than limit) must be provided and non-empty. Choose from: bounds, flights, callsigns, registrations, painted_as, operating_as, airports, routes, aircraft, altitude_ranges, squawks, categories, data_sources, airspaces, gspeed.' This clearly states when to use the tool (when you need real-time flight positions with filtering) and the mandatory requirement for at least one filter parameter, which is critical for correct invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.