Server Quality Checklist

Tool Scores

• book_lunchB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only explains parameters and does not mention side effects (e.g., whether booking overwrites an existing one), required permissions, or other behavioral traits like idempotency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a leading sentence and a list of arguments. However, it is somewhat verbose (8 lines) and could be more concise by merging some parameter explanations.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (7 parameters, 1 required) and existence of an output schema, the description adequately covers parameter semantics but lacks behavioral context such as success/error outcomes, idempotency, or constraints (e.g., lunch must be in the future).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description compensates for the 0% schema description coverage by explaining each parameter in detail, including format examples (e.g., date as DD-MM-YYYY) and notes on defaults (e.g., price of 0 keeps template price). This adds significant meaning beyond the schema's bare types and titles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Book office lunch for a date,' specifying the verb and resource. However, it does not explicitly differentiate from siblings like 'book_cab' or 'cancel_lunch', which would strengthen purpose clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus its siblings (e.g., cancel_lunch, list_lunch_bookings). There is no mention of prerequisites, constraints, or scenarios where the tool should not be used.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_lunch_bookingsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully convey behavioral traits. It indicates read-only and personal scope but doesn't clarify safety (side effects), rate limits, or what 'current and upcoming' exactly means (e.g., does it exclude past bookings?).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no unnecessary words. Every part adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, list-only operation) and the presence of an output schema, the description is nearly complete. It could be improved by noting ordering or time range, but adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters, so baseline 4 applies. The description adds no parameter info (none needed), but clearly states the tool's purpose. Schema coverage is 100% trivially.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and clearly identifies the resource ('your current and upcoming office lunch bookings'). It distinguishes from sibling tools like book_lunch and list_cab_bookings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for viewing personal bookings, but lacks explicit guidance on when to use versus alternatives (e.g., site-wide booking reports) or when not to use. No exclusions or context provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_cabA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only says 'Cancel', implying a state change, but does not describe side effects (e.g., refunds, reversibility), permissions required, rate limits, or endpoint behavior. The output schema exists but is not referenced to clarify return values.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: one line for purpose, then a docstring-style argument list. Every sentence adds value, no fluff. The key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 2 parameters and an output schema, the description covers the basic purpose and parameter sources. However, it lacks details on error handling, idempotency, or response structure, which are helpful for an agent to fully understand the tool's behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds essential meaning: booking_id is the trip ID from list_cab_bookings, and reason is optional. This clarifies the source and intent of parameters beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Cancel an office cab booking', specifying the verb (cancel) and the resource (cab booking). This distinguishes it from siblings like cancel_lunch and book_cab, which are different tools for different actions or resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains that the booking_id comes from list_cab_bookings, giving clear context on the prerequisite. However, it does not explicitly state when not to use this tool (e.g., if the booking is already cancelled) or provide alternatives beyond the sibling tool context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_lunchA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description only states 'cancel' with no details on side effects, permissions, or irreversibility. Lacks behavioral context beyond the action itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise: one-line purpose and parameter list. Front-loaded, no extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Output schema exists so return values not needed. However, lacks behavioral details (e.g., reversibility, confirmation) for a cancel action. Minimal but functional.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, description adds meaning: booking_id sourced from list_lunch_bookings, reason is optional. Compensates for missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'cancel' and resource 'lunch booking', distinctly different from sibling tools like 'book_lunch' and 'cancel_cab'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  States that booking_id comes from 'list_lunch_bookings', providing implicit guidance on prerequisite action. No explicit when-not-to-use, but clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• book_cabA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavioral traits. It only states that the tool books a cab, without mentioning any side effects, authentication requirements, or confirmation behavior. This lacks transparency for a mutation action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured with an Args section. Every sentence adds value, and there is no superfluous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists and parameters are well described, the description is mostly complete. It could mention prerequisites like user login or cab availability, but the core functionality is sufficiently covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds significant meaning beyond the schema by providing context for each parameter: pickup location type, drop location, ISO 8601 format for when, and clarification that notes are optional. This compensates for the 0% schema description coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'Book an office cab,' which is a clear verb+resource combination. It immediately distinguishes this tool from siblings like cancel_cab and list_cab_bookings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage through parameter explanations but does not explicitly state when to use this tool versus alternatives like book_lunch or cancel_cab. However, the context of cab vs lunch is clear from the tool name and siblings list.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_cab_bookingsA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It mentions 'current and upcoming' bookings but does not disclose other behaviors like pagination or data recency. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is clear and concise, with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters and an output schema present, the description provides adequate scope ('current and upcoming'). It is complete for this simple list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters, so schema coverage is complete. Baseline score of 4 is appropriate as no additional parameter information is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'office cab bookings'. It distinguishes from sibling tools like list_lunch_bookings and book_cab/cancel_cab by specifying 'cab bookings' and 'current and upcoming'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like list_lunch_bookings. The usage context is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.