Server Quality Checklist

Tool Scores

• check_availabilityC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool checks availability but doesn't describe what 'availability' means (e.g., time slots, table types), whether it's a read-only operation, potential rate limits, or error conditions. The 'mock implementation' note suggests limited real-world functionality but lacks specifics on behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose. The 'mock implementation' note adds context without verbosity. However, it could be more structured by explicitly separating purpose from implementation notes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, 100% schema coverage, and no output schema, the description is minimally adequate. It states the purpose but lacks details on return values (e.g., what availability data looks like) and behavioral context, which is needed given the absence of annotations. It meets basic requirements but leaves gaps in understanding tool behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't explain how parameters interact or typical values). Baseline 3 is appropriate as the schema handles parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'Check availability for a restaurant reservation' with the specific verb 'check' and resource 'availability'. It distinguishes from siblings like 'make_reservation' (which books) and 'search_restaurants' (which finds restaurants), though it doesn't explicitly differentiate from 'get_restaurant_details' which might provide related info.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'make_reservation' or 'search_restaurants'. It mentions 'mock implementation', which hints at testing contexts, but offers no explicit when/when-not instructions or prerequisites for effective tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_booking_instructionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves instructions but doesn't cover critical aspects like whether it's a read-only operation, potential rate limits, authentication needs, or what the return format looks like (e.g., text, structured data). This leaves significant gaps for an agent to use it effectively.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action ('Get instructions'), making it easy to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for effective tool use. It doesn't address behavioral traits (e.g., safety, performance) or output details, which are crucial for an agent to handle this tool correctly in a broader context with sibling tools like 'make_reservation'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('locale' and 'placeId') with their purposes and types. The description doesn't add any semantic details beyond what the schema provides, such as explaining how 'placeId' relates to restaurant selection or the impact of 'locale' on instruction content. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('instructions on how to make a reservation at a specific restaurant'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'check_availability' or 'make_reservation', which might involve related reservation processes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'check_availability' or 'make_reservation'. It lacks context about prerequisites (e.g., needing a restaurant ID) or exclusions, leaving the agent to infer usage based on the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• make_reservationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a 'mock implementation', which hints at limited functionality, but doesn't describe what that means operationally (e.g., whether it actually creates reservations, returns simulated results, or has specific limitations). For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core purpose without any wasted words. The parenthetical '(mock implementation)' is appropriately placed and adds necessary context without disrupting flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a mutation tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what happens after the 'attempt' (success/failure outcomes, return format, error conditions), nor does it address behavioral aspects like authentication needs or rate limits that would be crucial for an agent to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds no additional parameter information beyond what's in the schema, maintaining the baseline score of 3 for adequate but not enhanced parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('make') and resource ('restaurant reservation'), and the parenthetical '(mock implementation)' adds useful context about its nature. However, it doesn't explicitly distinguish this tool from its siblings like 'check_availability' or 'get_booking_instructions', which would be needed for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_availability' or 'get_booking_instructions'. It doesn't mention prerequisites (e.g., whether availability should be checked first) or appropriate contexts, leaving the agent to guess based on tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_restaurant_detailsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool retrieves information, implying a read-only operation, but lacks details on permissions, rate limits, error handling, or response format. This is a significant gap for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and lists key details without unnecessary words. Every element (verb, resource, information types) earns its place, making it appropriately sized and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no output schema, no annotations), the description is minimally adequate. It covers the purpose and data types but lacks behavioral context, usage guidelines, and output details, leaving gaps that reduce completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both parameters (placeId and locale). The description does not add any parameter-specific details beyond what the schema provides, such as examples or usage context, resulting in a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get detailed information') and resource ('about a specific restaurant'), specifying what information is retrieved (reviews, photos, opening hours). However, it does not explicitly differentiate from sibling tools like 'search_restaurants' or 'check_availability', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., needing a placeId from search results), exclusions, or comparisons to siblings like 'search_restaurants' for finding restaurants or 'check_availability' for availability details.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_restaurantsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It discloses key behavioral traits: returns top 3 AI-recommended restaurants, operates within 3km radius (implied default), and uses Google API for locale-specific results. However, it doesn't mention rate limits, authentication needs, error conditions, or whether this is a read-only operation (though 'search' implies it).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences that efficiently cover purpose and additional capability (keyword search). The first sentence front-loads core functionality with key parameters and output details. No wasted words, though it could be slightly more structured by separating constraints from capabilities.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 10-parameter search tool with no annotations and no output schema, the description provides adequate but incomplete context. It covers what the tool does and key constraints (top 3, 3km radius), but lacks details about return format, error handling, or how AI recommendations work. The schema compensates for parameter documentation, but behavioral aspects remain partially uncovered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 10 parameters thoroughly. The description adds marginal value by mentioning location, cuisine types, mood, event type, and keywords as search criteria, but doesn't provide additional syntax or format details beyond what's in the schema descriptions. Baseline 3 is appropriate when schema does heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for restaurants based on multiple criteria (location, cuisine types, mood, event type, keywords) and returns top 3 AI-recommended results within a 3km radius. It specifies the verb 'search' and resource 'restaurants' with scope details, though it doesn't explicitly differentiate from sibling tools like 'get_restaurant_details'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for finding restaurants with various filters, but doesn't explicitly state when to use this tool versus alternatives like 'get_restaurant_details' or 'check_availability'. It mentions 'you can also search for specific food types using keywords', which provides some context but lacks clear exclusions or comparisons to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.