Server Quality Checklist

Tool Scores

• get_booking_by_idC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves 'detailed information' but doesn't specify what that includes (e.g., booking details, status, timestamps), whether it's a read-only operation, potential error conditions (e.g., invalid ID), or performance aspects like rate limits. This leaves significant behavioral gaps for the agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise and front-loaded, with the core purpose stated first followed by parameter details. The two-sentence structure is efficient, though the 'Args:' section could be integrated more smoothly. There's no unnecessary verbosity, earning a high score for clarity and brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with one parameter) and lack of annotations or output schema, the description is incomplete. It doesn't cover what information is returned, error handling, or dependencies on other tools (e.g., needing a valid booking ID from 'create_booking'). For a tool with no structured data support, more context is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds minimal semantics beyond the input schema, which has 0% description coverage. It clarifies that 'booking_id' is a 'unique ID of the booking', but doesn't explain format constraints (e.g., numeric range) or provide examples. Since the schema lacks descriptions, the baseline is low, and the description only partially compensates, resulting in an average score.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('detailed information about a specific booking'), distinguishing it from sibling tools like 'get_bookings' (which likely lists multiple bookings). However, it doesn't explicitly differentiate from 'get_property_by_id' or 'update_booking_status' in terms of resource type, which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose 'get_booking_by_id' over 'get_bookings' (e.g., for a single booking vs. multiple), nor does it address prerequisites like authentication or availability. This lack of contextual direction leaves the agent without clear usage cues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_propertiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'Get a list' implying a read operation, but doesn't cover critical aspects like authentication requirements, rate limits, pagination behavior beyond limit/offset, error conditions, or response format. The description is minimal and leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with a clear purpose statement followed by parameter documentation. The Args section is well-structured with one line per parameter. No redundant information or unnecessary elaboration is present. It could be slightly more front-loaded with usage context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and 3 parameters, the description is incomplete. While it documents parameters, it lacks information about the return format, error handling, authentication, and when to use versus siblings. For a list operation tool with filtering capabilities, more contextual information would be expected.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides parameter documentation in an Args section, explaining limit, offset, and status with examples. However, schema description coverage is 0%, so the description compensates by documenting all parameters. It adds meaning beyond the bare schema by explaining defaults and providing status examples, but doesn't elaborate on constraints or advanced usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get a list of properties with optional filtering.' This specifies the verb ('Get'), resource ('properties'), and scope ('list with filtering'), distinguishing it from sibling tools like get_property_by_id. However, it doesn't explicitly differentiate from other list operations like get_bookings, so it falls short of a perfect 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions optional filtering but doesn't specify scenarios for using status filtering or when to choose this over get_property_by_id for single properties. No exclusions, prerequisites, or sibling comparisons are included.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_property_by_idC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but adds minimal context. It implies a read-only operation ('Get') but doesn't cover aspects like authentication needs, rate limits, error handling, or response format. This is inadequate for a tool with zero annotation coverage, leaving significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with the core purpose stated first in a clear sentence. The additional parameter explanation is brief and relevant, though it could be more integrated. There's minimal waste, but the structure is slightly basic.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It covers the basic purpose and parameter but misses critical context such as what 'detailed information' includes, error scenarios, or behavioral traits. For a tool with no structured support, this leaves the agent under-informed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds basic meaning for the single parameter ('property_id: The unique ID of the property'), which is helpful since schema description coverage is 0%. However, it doesn't provide details like ID format, range, or examples, offering only marginal value beyond what the schema's type ('integer') already indicates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('detailed information about a specific property'), making it immediately understandable. However, it doesn't differentiate from sibling tools like 'get_properties' (which likely lists multiple properties) or 'get_booking_by_id' (which focuses on bookings rather than properties), missing full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. For example, it doesn't specify when to choose 'get_property_by_id' over 'get_properties' (e.g., for single vs. multiple properties) or other siblings, leaving the agent to infer usage from context without explicit direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bookingsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Get' implies a read operation, it doesn't specify whether this requires authentication, has rate limits, returns paginated results (though parameters suggest it), or what format the bookings data takes. The description mentions filtering but doesn't explain behavioral aspects like default ordering, error conditions, or response structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement followed by a well-organized parameter list. Each parameter explanation is brief but informative. There's no wasted text, though the formatting could be slightly improved (note the inconsistent indentation of 'end_date'). Overall, it's appropriately sized for a 6-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read operation with 6 parameters and no annotations or output schema, the description is moderately complete. It covers all parameters well but lacks behavioral context (authentication, rate limits, response format). The absence of output schema means the description should ideally mention what 'bookings' data looks like, but it doesn't. It's adequate but has clear gaps for agent invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates well by providing clear explanations for all 6 parameters. It adds meaningful context beyond the schema: explaining 'size' as 'maximum number', 'page' for pagination, 'property_id' for filtering, 'status' with examples, and date formats for 'start_date'/'end_date'. This significantly enhances understanding beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Get bookings with optional filtering', which is a specific verb+resource combination. It distinguishes itself from siblings like 'get_booking_by_id' (single booking) and 'get_calendar'/'get_occupancy_summary' (different data types), though it doesn't explicitly mention these distinctions in the description text itself.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose this over 'get_booking_by_id' for single bookings, or how it differs from 'get_calendar' or 'get_occupancy_summary' for related data. There's no discussion of prerequisites, limitations, or appropriate contexts for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_calendarB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool gets information, implying a read-only operation, but doesn't cover critical aspects like authentication needs, rate limits, error handling, or the format of returned data. For a tool with 4 parameters and no output schema, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by a clear, bullet-like list of parameters with brief explanations. Every sentence earns its place by providing essential information without redundancy or fluff, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (4 parameters, no annotations, no output schema), the description is incomplete. It covers parameter semantics well but lacks behavioral context (e.g., data format, error cases) and usage guidelines. Without an output schema, the description should ideally hint at return values, but it doesn't, leaving gaps in understanding how to interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds substantial meaning beyond the input schema, which has 0% description coverage. It explains each parameter's purpose (e.g., 'property_id: The property ID', 'start_date: Start date for calendar (YYYY-MM-DD)'), including format hints for dates. This compensates well for the schema's lack of descriptions, though it doesn't detail default behaviors for optional parameters like room_type_id.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get calendar/availability information for a property.' It uses a specific verb ('Get') and identifies the resource ('calendar/availability information for a property'), making the function evident. However, it doesn't explicitly differentiate from sibling tools like 'get_occupancy_summary' or 'get_bookings', which might also involve calendar-related data, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_occupancy_summary' or 'get_bookings', nor does it specify contexts or exclusions for usage. This lack of comparative information leaves the agent without clear direction for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_booking_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is an update operation, implying mutation, but doesn't cover permissions needed, whether changes are reversible, error conditions (e.g., invalid status), or response format. This leaves significant gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose in the first sentence, followed by a concise Args section. Every sentence adds value without redundancy, making it efficient and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (mutation with 2 parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't explain behavioral aspects like side effects, error handling, or return values, which are critical for safe and effective use by an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaningful context: 'booking_id' is explained as 'The unique ID of the booking', and 'status' includes examples ('Booked', 'Cancelled', etc.). This clarifies parameter purposes beyond the schema's basic types, though it could specify format constraints (e.g., status enum).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Update') and resource ('status of an existing booking'), making the purpose specific and understandable. However, it doesn't explicitly differentiate from sibling tools like 'create_booking' or 'get_booking_by_id' beyond the implied update vs. create/read distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., booking must exist), exclusions, or comparisons to siblings like 'create_booking' for new bookings or 'get_booking_by_id' for viewing status.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_occupancy_summaryB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes a read operation ('Get') but doesn't cover critical aspects like authentication requirements, rate limits, error handling, or data freshness. For a tool with three parameters and no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement followed by parameter explanations. Every sentence adds value: the first defines the tool's function, and the subsequent lines clarify parameter meanings. There's no redundant or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and parameters but lacks details on output format, error conditions, and behavioral traits. Without annotations or output schema, more context would be helpful for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful semantics beyond the input schema, which has 0% description coverage. It explains that 'property_id' identifies the property, and 'start_date' and 'end_date' define the analysis date range with format 'YYYY-MM-DD'. This compensates well for the schema's lack of descriptions, though it doesn't detail constraints like valid date ranges or property ID formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get occupancy summary, average rental rate, and total revenue for a property within a specified date range.' This specifies the verb ('Get'), resource ('property'), and outputs (summary, rate, revenue). However, it doesn't explicitly differentiate from siblings like 'get_properties' or 'get_property_by_id' beyond the specific metrics it retrieves.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, compare to sibling tools like 'get_bookings' or 'get_calendar', or specify scenarios where this summary is preferred over detailed data. The usage context is implied by the parameters but not explicitly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_bookingB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Create' implies a write operation, the description doesn't address critical aspects like required permissions, whether the operation is idempotent, error handling, or what happens on success/failure. For a mutation tool with 13 parameters, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement followed by a comprehensive parameter list. Every sentence serves a purpose, though the parameter explanations could be slightly more concise in places. The front-loaded purpose statement helps the agent quickly understand the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (13 parameters, mutation operation) and lack of both annotations and output schema, the description is moderately complete. It excels at parameter semantics but lacks behavioral context, usage guidelines, and information about what the tool returns. For a creation tool with many parameters, more comprehensive guidance would be beneficial.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides excellent parameter semantics beyond the schema. For all 13 parameters, it adds meaningful explanations (e.g., 'Arrival date (YYYY-MM-DD)', 'Guest's full name', 'Currency code (default: "USD")') that clarify purpose and format. With 0% schema description coverage, this fully compensates and provides essential context the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Create') and resource ('booking'), making it immediately understandable. However, it doesn't differentiate this from sibling tools like 'update_booking_status' or explain how it relates to read operations like 'get_bookings', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'update_booking_status' or how it relates to read operations. There's no mention of prerequisites, constraints, or typical use cases, leaving the agent to infer usage from context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.