Server Quality Checklist

Tool Scores

• cancel_bookingB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions the effect on seat availability, but does not explain what happens to the booking record, whether the operation is idempotent, what error conditions exist, or any authentication requirements. For a mutation tool, this is a minimal disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence with no redundant information. It is front-loaded with the core action and outcome, making it efficient for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sparse for a tool with no annotations and no output schema. It does not mention what happens on success (e.g., confirmation, return value), how to handle invalid references, or what state changes occur beyond seat availability. An agent would need additional context to call it correctly and interpret the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage, and the description does not explain the booking_reference parameter at all. It gives no indication of the expected format, source, or context of this identifier. The agent is left entirely to infer its purpose from the parameter name alone, which is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('cancel'), a resource ('booking'), and an explicit outcome ('restore the seat availability'). It clearly distinguishes itself from siblings like book_flight and get_booking, leaving no ambiguity about its function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied by the verb 'cancel' and the sibling list, but the description does not explicitly state when to use this tool versus alternatives, nor does it mention any prerequisites or exclusion conditions. It provides no direct guidance beyond the inherent meaning of the operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_flightsB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the full burden. It honestly discloses that results are fictional and not connected to live availability, which is valuable. However, it omits other behavioral details like error handling, pagination, or read-only status. Given that the key caveat is present, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no redundancy. The action is front-loaded, and the demo disclaimer is clearly stated. Every word adds value, making it a model of conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with an output schema, the description covers the core purpose and the crucial fake-data caveat. However, it lacks guidance on when to use this tool versus fetching a specific flight (get_flight), and does not mention search result behavior. This is adequate but with clear gaps, so a 3 is justified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no parameter-level meaning beyond what the parameter names already suggest (origin, destination, date, etc.). The description does not compensate for the lack of schema descriptions, so it earns a 2.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Search') and a specific resource ('the local sample flight inventory'), and clarifies the fictional/demo nature. It distinguishes itself from live booking systems but does not explicitly differentiate from sibling tools like get_flight, so it earns a 4 rather than a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the siblings (e.g., get_flight for a specific flight). The 'local sample' phrasing implies a demo/testing context, but there is no explicit when-to-use or when-not-to-use guidance, resulting in a score of 2.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_seatsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It only says 'Check currently available seats', which implies a read-only operation but does not disclose side effects, return format, error conditions, or rate limiting. For a tool with zero annotation coverage, this is insufficiently transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that conveys the essential action without fluff. Every word earns its place, and the core purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter read tool with no output schema, the description provides the core functionality. It misses a note on return values (e.g., count vs. list) and any usage context, but is largely adequate for an agent to call correctly. The absence of annotations and output schema makes it slightly incomplete, but not severely.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no description for flight_id (0% coverage), so the description must clarify it. The phrase 'for a demo flight' adds a qualifier but does not explicitly state that flight_id identifies the flight. It partially compensates by linking the parameter to the flight context, but omits details like expected format or examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the task: checking currently available seats for a flight, with a specific verb and resource. However, it does not explicitly differentiate from sibling tools like get_flight or book_flight, though the involvement of seats is unique enough that this is minor.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites (e.g., must have a valid flight ID) or context like 'use before booking'. The intended use is only implied by the verb 'check'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bookingA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The only behavioral information is the word 'simulated', which describes the data nature, not the operation. It does not disclose read-only status, error behavior, or side effects. With no annotations, the description carries the full burden and falls short.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One sentence, front-loaded with the action and scope, zero waste. The format example is embedded efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter retrieval tool, it is mostly adequate, but it omits any mention of the return shape or error cases (e.g., not-found behavior). Given no output schema and no annotations, a bit more detail would help, but the low complexity mitigates the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description provides a concrete example format 'FLM-XXXXXX' which gives the agent a template for the booking_reference parameter, adding value beyond the schema's bare type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb (Retrieve), resource (booking), and method (by reference code). It clearly distinguishes from book_flight and cancel_booking based on the action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implies usage when you have a booking reference code, but does not explicitly contrast with siblings or mention when not to use it. The condition (having a reference) is clear enough, but no alternatives are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_flightA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates a read operation ('Retrieve') and adds 'fictional' to set expectations about data context, but it doesn't disclose error behavior, response format, or any side effects beyond that. For a simple get-by-id tool, this is adequate but minimal; it doesn't go beyond what the name implies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently communicates the tool's purpose without redundant words. Every word adds value, and it is immediately clear what the tool does.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, the description should explain what 'complete information' entails and any prerequisites. It says the flight is fictional, which sets expectations, but it doesn't describe the structure of the returned flight data or any potential errors (e.g., not found). For a tool with one simple parameter, this is minimally acceptable but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% coverage for the only parameter, flight_id. The description says 'by its ID,' which clarifies that flight_id is the identifier, but it doesn't provide format, constraints, or an example. This partially compensates for the schema gap but doesn't fully clarify what values are acceptable or how to obtain a valid ID (e.g., via search_flights).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Retrieve'), a resource ('flight'), and the key discriminator (by ID). This clearly distinguishes it from sibling tools like search_flights (which finds flights) and get_booking (which retrieves bookings). No ambiguity exists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: when you have a flight_id and want its complete information. It doesn't explicitly mention alternatives or exclusions, but the context is clear enough that an agent would know to use this over search_flights when an identifier is already known. However, it lacks an explicit statement like 'use this when you have an ID, use search_flights to find flights.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• book_flightA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly states the simulation nature and lack of real payment, which is crucial for agents to avoid misunderstanding. It does not describe side effects such as persistence details, return values, or error conditions, but the key behavioral caveat is clearly disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The core action and destination are front-loaded, and the critical caveat about simulation follows immediately. That is optimal structure for an agent scanning rapidly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with 5 parameters, no annotations, and no output schema, the description is adequate but incomplete. It tells what it does and the confirmation requirement, but does not mention expected return values, handling of invalid flight_id, or any side effects beyond the simulation note. An agent would likely need to infer or make extra calls to handle errors.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameter meanings. It mentions zero parameters. Although the names (flight_id, passenger_name, passenger_email, passenger_phone, seat_number) are self-explanatory, the description adds no semantic value beyond the schema, failing to clarify requiredness, formats, or relationships between parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb-object pair: 'Create a simulated flight booking in the local SQLite database.' It immediately distinguishes itself from a real purchase by stating 'This does NOT purchase a real airline ticket or process payment.' This specificity differentiates it from siblings like search_flights, get_booking, and cancel_booking, making the tool's role unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit precondition: 'Use this ONLY after receiving explicit user confirmation to book.' This is a clear 'when-to-use' rule. However, it does not mention when not to use (beyond the implied no-real-purchase) or point to alternative tools for scenarios like searching or checking seats, so it lacks the full set of alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.