Server Quality Checklist

Tool Scores

• search_flightsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions the prerequisite session requirement, which is valuable context. However, it doesn't describe what the tool returns (no output schema exists), whether it's idempotent, rate limits, error conditions, or what happens if the session token is invalid. For a search tool with complex parameters, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that each serve a clear purpose: stating the tool's function and providing prerequisite guidance. There's no wasted text. The minor typo ('exexcuted') doesn't significantly impact clarity. It could be slightly more structured but efficiently conveys essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 parameters with nested objects, no output schema, no annotations), the description provides basic but incomplete context. It covers the prerequisite requirement well but doesn't explain what the search returns, how results are structured, or behavioral aspects like pagination or error handling. For a flight search tool that presumably returns complex results, this leaves important gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning all parameters are documented in the schema itself. The description doesn't add any parameter-specific information beyond what's already in the schema descriptions. It mentions the session token requirement, but this is already covered in the schema. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Perform a flight search based on search criteria.' This specifies the verb ('search') and resource ('flights'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'create_cart' or 'initialize_booking_session' beyond mentioning the latter as a prerequisite.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'This operation required 'initialize_booking_session' to be exexcuted first to start the session.' This clearly states a prerequisite and when to use this tool (after initialization). It doesn't mention alternatives, but the prerequisite guidance is specific and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• initialize_booking_sessionA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions that this creates a session token for subsequent steps, which is useful context, but lacks details on error handling, rate limits, authentication needs, or what happens if parameters are invalid. It provides some behavioral insight but not comprehensive coverage for a session-initialization tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with just one sentence ('Initialize a new booking session.') and the schema includes a detailed description that adds necessary context without redundancy. Every part earns its place, making it front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (nested objects, 1 parameter with rich schema) and no output schema, the description is reasonably complete. It explains the tool's role as a first step and hints at the response (session token), but could be more detailed about behavioral aspects like error cases or session management, given the lack of annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal parameter semantics beyond the schema, such as emphasizing it's for 'initial flight search parameters,' but doesn't provide additional context like examples or edge cases. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Initialize') and resource ('booking session'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'create_cart' or 'search_flights', which might be related booking operations, so it doesn't achieve full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states 'REQUIRED FIRST STEP' and provides clear guidance on when to use it ('Call this endpoint first to receive a session token, then use that token for actual flight searches'), including the sequence of operations and the alternative tools to use next.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_cartA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and does so effectively. It explains that the cart validates selections, checks availability, calculates final pricing, and returns a cartId required for subsequent operations. However, it doesn't mention potential errors, rate limits, or authentication details beyond the session-token, leaving some behavioral aspects uncovered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (PREREQUISITES, WORKFLOW POSITION, RESPONSE, IMPORTANT) and front-loaded key information. It's appropriately sized, but some redundancy exists (e.g., repeating the importance of recent search_flights data), which slightly reduces efficiency without major waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a multi-step booking flow with no annotations and no output schema, the description does a good job explaining prerequisites, workflow position, and response. However, it lacks details on error handling, session expiration, or what happens if availability changes, which would be helpful for a tool with mutation implications.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters thoroughly. The description adds some context by emphasizing that 'airBoundIds must come from the most recent search_flights response within the same session', but this is partially covered in the schema. It doesn't provide additional semantic meaning beyond what the schema offers, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Create a shopping cart with selected flight options') and resource ('cart'), distinguishing it from sibling tools like 'initialize_booking_session' and 'search_flights' by explaining its role in the booking flow. It explicitly mentions the tool adds selected flights to a cart for booking, which is distinct from session initialization or flight searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'PREREQUISITES: Must call 'initialize_booking_session' first, then 'search_flights' to get available flight options' and 'WORKFLOW POSITION: Step 3 of the booking flow (after initialization and search, before passenger details and payment)'. It also specifies an alternative by referencing the sibling tools, making it clear this is not the first step in the workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.