Server Quality Checklist

Tool Scores

• get_supplier_infoA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It describes the tool as returning information, which implies a read-only operation, but doesn't explicitly state whether it's safe, whether it requires authentication, or any rate limits. The description adds some context about what information is returned (destinations and experience types) but lacks detailed behavioral traits. This is adequate but has clear gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose followed by usage guidance. Every sentence earns its place by adding clear value, with zero waste or redundancy. It's appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is complete enough. It explains what the tool does and when to use it, which covers the essential context. However, it could benefit from more detail on return values or behavioral aspects, but for a read-only info tool, this is largely sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100% (though empty). The description doesn't need to add parameter semantics, so it naturally compensates by focusing on usage. Baseline for 0 parameters is 4, as the description provides value without parameter details.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Returns information about the supplier network and available inventory.' It specifies both the resource ('supplier network and available inventory') and the action ('returns information'), though it doesn't explicitly distinguish from siblings beyond mentioning one alternative. The purpose is clear but sibling differentiation is only partial.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Use this to understand what destinations and experience types are available before calling search_slots.' It clearly states when to use this tool (before search_slots) and names a specific alternative (search_slots), which meets the criteria for a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_booking_statusA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool as a read-only status check (implied by 'Check'), and details the possible status values with explanations, including time estimates (e.g., 'up to 45s' for fulfilling) and outcomes (e.g., 'payment hold cancelled' for failed). This adds valuable context beyond basic functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. Each sentence earns its place: the first states the action, the second explains the parameter, the third describes the return, and the list details status values without redundancy. It is efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no output schema, no annotations), the description is complete. It covers the purpose, parameter semantics, return values, and status details, providing all necessary information for an agent to use the tool correctly without gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the input schema, which has 0% description coverage. It explains that booking_id is 'returned by book_slot', clarifying its origin and purpose. This compensates fully for the schema's lack of documentation, providing essential context for the single parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Check') and resource ('status of a booking'), and distinguishes it from siblings like book_slot (which creates bookings) and search_slots (which finds available slots). It directly addresses what the tool does without being vague or tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by referencing book_slot as the source for booking_id, implying this tool should be used after a booking is created. However, it does not explicitly state when not to use it or mention alternatives like get_supplier_info for supplier details, which could help avoid misuse.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_slotsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: it returns 'real production inventory' from 19 specific suppliers, uses 'live' sourcing via OCTO protocol, sorts by urgency (soonest first), and explains result sorting logic. It doesn't mention rate limits or authentication requirements, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with purpose statement, supplier list, sorting explanation, parameter documentation, and return format - all in appropriate detail. Every sentence adds value, and it's well-organized with clear sections for Args and Returns.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, no annotations, but has output schema), the description is remarkably complete. It covers purpose, behavioral context, detailed parameter semantics, and explains the return format. With an output schema present, it appropriately doesn't need to detail return value structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantics for all 5 parameters: explains city accepts partial matches and can be empty, specifies category should be 'experiences' for tours/activities, clarifies hours_ahead default and meaning, explains max_price=0 means all prices, and provides nuanced guidance on limit usage with sorting implications.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for 'last-minute available tours and activities' with specific details about real inventory from 19 suppliers via OCTO protocol. It distinguishes from siblings like book_slot (booking) and get_booking_status (status checking) by focusing on search functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (searching for last-minute tours/activities) and includes practical guidance like 'Use city/category filters to narrow results instead of raising the limit when possible.' However, it doesn't explicitly state when NOT to use it or mention alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• book_slotA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does an excellent job disclosing behavioral traits: it explains the two distinct modes, payment flow differences (Stripe Checkout vs. pre-funded wallet), session expiration (24 hours), and immediate action requirements (sharing URL directly). It doesn't cover rate limits or error handling specifics, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sectioning (two modes, args, returns), uses bullet points for readability, and every sentence earns its place by providing essential operational guidance. It's appropriately sized for a complex tool with multiple modes and avoids redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, two distinct modes, payment integration) and absence of both annotations and output schema, the description provides complete context: it explains both operational modes, documents all parameters, specifies return formats for both success cases and errors, and provides necessary implementation guidance. No significant gaps remain.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining all 7 parameters: it clarifies slot_id's source (search_slots results), provides format examples for customer_phone, explains quantity's effect on pricing, and crucially details the interaction between wallet_id and execution_mode that triggers autonomous mode. This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('Book a last-minute slot for a customer') and distinguishes it from siblings by focusing on booking functionality rather than searching (search_slots), checking status (get_booking_status), or retrieving supplier info (get_supplier_info). The two operational modes further clarify the specific actions involved.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use each mode: approval mode (default, no wallet_id) for customer-facing payment flows, and autonomous mode (wallet_id + execution_mode='autonomous') for automated payment scenarios. It clearly distinguishes between the two alternatives and specifies the conditions for each.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.