Server Quality Checklist

Tool Scores

• flightHappinessIndexC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It doesn't disclose any behavioral traits: no indication of whether this is a read-only query, if it has side effects, rate limits, authentication needs, or what the output looks like. The description only lists topics without explaining how the tool behaves when invoked. This is inadequate for a tool with 4 parameters and no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single run-on sentence listing topics, which is somewhat concise but poorly structured. It starts with a usage hint but lacks clear organization. While it avoids unnecessary words, the list format with 'etc.' at the end feels incomplete and could be better formatted for readability. It's front-loaded with the usage context, but the content is vague.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, no annotations, no output schema), the description is incomplete. It doesn't explain what the tool returns, how the 'happiness index' is calculated or presented, or any behavioral aspects. The list of topics is broad but doesn't tie back to the input parameters or expected output. For a tool presumably providing detailed flight assessments, this leaves too many gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents all 4 parameters (arr, date, dep, fnum) with clear descriptions and patterns. The description adds no parameter-specific information beyond what's in the schema. According to guidelines, when schema coverage is high (>80%), the baseline is 3 even with no param info in the description, which applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description lists topics but doesn't state what the tool actually does. It says 'using this tool when you need information related to following topics' followed by a list, but doesn't specify the action (e.g., 'retrieve', 'calculate', 'compare'). It's vague about whether this returns a happiness index score, provides detailed reports, or something else. The name 'flightHappinessIndex' suggests some kind of scoring or rating, but the description doesn't confirm this.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description starts with 'using this tool when you need information related to following topics' which provides some context, but doesn't explicitly say when to use this vs. sibling tools like searchFlightItineraries or searchFlightsByNumber. It lists broad categories but gives no guidance on prerequisites, alternatives, or exclusions. For example, it doesn't clarify if this is for specific flights (matching the schema) or general comparisons.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchFlightItinerariesB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions searching for 'purchasable flight options and the lowest price', which implies read-only querying, but does not address critical behaviors such as rate limits, authentication needs, error handling, or what the output format looks like (e.g., list of flights, pricing details). This leaves significant gaps for an agent to understand how to interpret results.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Search for purchasable flight options and the lowest price') followed by parameter details. The parenthetical examples are relevant but slightly lengthen the text. Overall, it is well-structured with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of flight search (multiple parameters, no output schema, and no annotations), the description is incomplete. It lacks information on output format (e.g., what data is returned), error conditions, or behavioral constraints like pagination or availability. This makes it inadequate for an agent to fully understand how to use the tool effectively in practice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter well-documented in the input schema (e.g., patterns, examples). The description adds minimal value beyond the schema by reiterating the three-letter code examples (BJS, SHA, etc.) and mentioning the departure date, but does not provide additional semantic context or usage nuances. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Search for purchasable flight options and the lowest price') and identifies the key resources (departure city, arrival city, departure date). It distinguishes this tool from siblings like 'searchFlightsByDepArr' by emphasizing price comparison and purchasability, which is not mentioned in other tool names.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through its parameter examples (e.g., BJS for Beijing), but does not explicitly state when to use this tool versus alternatives like 'searchFlightsByDepArr'. It provides some context for date handling (referring to 'getTodayDate' tool), but lacks clear exclusions or comparisons with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getFlightTransferInfoB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It adds useful behavioral context: the date format requirement (YYYY-MM-DD), the rule about using 'getTodayDate' for today's date, and the IATA code requirement. However, it doesn't disclose other important behaviors like whether this is a read-only operation, potential rate limits, error conditions, or what 'flight transfer info' specifically includes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with three sentences. The first sentence states the purpose, the second provides format requirements, and the third gives important usage rules. Each sentence earns its place, though the structure could be slightly improved by front-loading the most critical information more clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, 100% schema coverage, and no annotations or output schema, the description is moderately complete. It covers basic usage rules and format requirements but lacks details about what 'flight transfer info' returns, error handling, or how this differs from sibling flight search tools. The absence of an output schema increases the burden on the description, which it doesn't fully meet.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all three parameters thoroughly. The description adds minimal value beyond the schema: it reiterates the date format and IATA code requirements, but doesn't provide additional semantic context about what 'flight transfer info' means or how parameters interact. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get flight transfer info by departure city and arrival city and departure date.' It specifies the verb ('Get') and resources ('flight transfer info'), but doesn't explicitly differentiate from sibling tools like 'searchFlightItineraries' or 'searchFlightsByDepArr' which might have overlapping functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some usage guidance: it mentions using 'getTodayDate' for today's date instead of hardcoding, which is a specific context rule. However, it doesn't explain when to use this tool versus alternatives like 'searchFlightsByDepArr' or 'searchFlightItineraries', leaving the agent to infer based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchFlightsByNumberA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds some useful context about date handling requirements (using getTodayDate for today's date) and airport code format. However, it doesn't disclose important behavioral aspects like what happens if no flights are found, whether this is a read-only operation, rate limits, or authentication requirements. The description provides some operational constraints but misses key behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose. Each sentence adds value: format examples, optional parameters, date format, and important usage rule. There's no wasted text, though the structure could be slightly improved by grouping related information more clearly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 4 parameters, 100% schema coverage, but no annotations and no output schema, the description provides adequate but incomplete context. It covers the basic operation and some constraints, but doesn't explain what the tool returns (no output schema) or address potential error conditions. Given the complexity and lack of structured output information, the description should do more to compensate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema: it reiterates the flight number format example and mentions that dep/arr are optional. However, it doesn't provide additional semantic context about parameter interactions or edge cases not covered in the schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search flights by flight number and date.' It specifies the verb ('search') and resource ('flights'), but doesn't explicitly differentiate from sibling tools like 'searchFlightsByDepArr' or 'searchFlightItineraries' beyond the 'by flight number' aspect. The description is specific about what it searches by, but lacks sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: when searching by flight number and date. It explicitly mentions an alternative tool ('getTodayDate') for obtaining today's date, which is helpful guidance. However, it doesn't specify when NOT to use this tool versus other flight search siblings like 'searchFlightsByDepArr' or 'searchFlightItineraries'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getFutureWeatherByAirportA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It specifies the forecast duration and IATA code format but lacks details on permissions, rate limits, error handling, or response format. For a tool with zero annotation coverage, this leaves significant gaps in understanding its operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the tool's purpose and scope, and the second clarifies the parameter with examples. Every sentence adds essential information without redundancy, making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter, no output schema, no annotations), the description is adequate but incomplete. It covers the purpose and parameter semantics well but lacks behavioral details like response format or error conditions, which are important for a weather forecasting tool without structured output documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents the parameter fully. The description adds value by reinforcing the IATA code requirement and providing examples (PEK, SHA, CAN, HFE), which aids in understanding the parameter's practical use beyond the schema's pattern constraint.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get airport future weather'), resource ('airport'), and scope ('for 3 days (today, tomorrow, the day after tomorrow) by airport IATA 3-letter code'). It distinguishes itself from sibling tools by focusing on weather forecasting rather than flight data, date, or location queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying the 3-day forecast and IATA code requirement, but it does not explicitly state when to use this tool versus alternatives (e.g., for weather planning vs. flight tracking). No exclusions or direct comparisons to sibling tools are provided, leaving some ambiguity about optimal use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getRealtimeLocationByAnumA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes the core behavior (retrieving realtime location) but doesn't disclose important traits like whether this is a read-only operation, potential rate limits, authentication requirements, or what happens when aircraft number is invalid. The description adds basic context but lacks comprehensive behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. First sentence states purpose, second provides crucial usage guidance. Every word earns its place, and the most important information (purpose) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read operation with no output schema, the description provides good context about purpose and usage guidelines. However, without annotations or output schema, it could better explain what the return value contains (e.g., coordinates, altitude, timestamp) and any limitations. The completeness is strong but not perfect given the lack of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the parameter type and format. The description adds minor clarification about the format ('Aircraft registration number like B2021, B2022, B2023, etc.') but doesn't provide additional semantic context beyond what's in the schema. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('flight realtime location') with specific qualification ('by aircraft number'). It distinguishes from siblings by focusing on realtime location retrieval rather than search, transfer info, or weather tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly provides when-not-to-use guidance: 'if aircraft number is unknown, you should try to request it using searchFlightsByNumber tool.' This names a specific alternative tool for the prerequisite step, giving clear context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchFlightsByDepArrA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: the requirement for valid IATA codes, date format constraints, and the interaction with the getTodayDate tool for date handling, though it lacks details on output format, error handling, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with each sentence earning its place by providing essential information without redundancy. It efficiently covers purpose, parameter usage, date handling, and code validation in a structured manner.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of 5 parameters, no annotations, and no output schema, the description does well by covering parameter semantics, usage guidelines, and behavioral aspects like date handling. However, it lacks details on the return format (e.g., what data is included in flight results), which is a minor gap in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the baseline is 3. The description adds significant value by explaining the semantic difference between dep/arr (airport codes) and depcity/arrcity (city codes), providing usage rules for cities with multiple airports, and reinforcing date format and tool dependencies, elevating it above the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Search for flights') and resource ('between airports or cities by date'), and distinguishes it from sibling tools like 'searchFlightItineraries' and 'searchFlightsByNumber' by specifying it searches by departure/arrival locations rather than itineraries or flight numbers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use specific parameters (depcity/arrcity vs. dep/arr based on city airport multiplicity) and when to use an alternative tool ('For today's date, use the getTodayDate tool'), offering clear context and exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getTodayDateA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it returns today's date in a specific format and timezone. However, it doesn't mention potential edge cases (e.g., timezone handling during daylight saving) or error conditions, leaving some behavioral aspects unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two sentences) and front-loaded with the core functionality. Every sentence earns its place: the first defines the tool's output, and the second provides critical usage guidance. There is zero wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (no parameters, simple output), no annotations, and no output schema, the description is complete enough. It specifies the exact format and timezone of the return value, and the usage guidance covers when to invoke it. For this simple tool, no additional context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% coverage with a single dummy parameter, so the schema already documents it. The description correctly indicates this is essentially a no-parameter tool by focusing on the date retrieval without mentioning inputs. It adds value by clarifying the tool's parameterless nature implicitly through its usage focus.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('Get') and resource ('today's date'), with precise format details ('YYYY-MM-DD format') and timezone context ('local timezone'). It distinctly differentiates from sibling tools which are all flight-related, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this tool whenever you need today's date' and 'NEVER hardcode dates.' This gives clear when-to-use instructions and a specific prohibition, helping the agent choose this tool over manual date handling or other date-related tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.