Server Quality Checklist

Tool Scores

• cancel_reservationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While 'Cancel' implies a destructive mutation, the description doesn't specify whether this action is reversible, requires specific permissions, has side effects (e.g., fees, notifications), or what happens on success/failure. For a mutation tool with zero annotation coverage, this leaves critical behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's front-loaded with the core action and resource, making it easy to parse. Every word earns its place, and there's no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a destructive mutation with no annotations and no output schema), the description is incomplete. It lacks information on behavioral traits, success/failure outcomes, error handling, and any constraints (e.g., cancellation policies). While the schema covers parameters well, the overall context for safe and effective use is insufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with clear documentation for both parameters ('reservation_id' and 'platform'). The description adds no additional meaning beyond what the schema provides—it doesn't explain parameter interactions, format details, or usage examples. With high schema coverage, the baseline score of 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Cancel') and resource ('an existing reservation'), making the tool's purpose immediately understandable. It distinguishes from siblings like 'make_reservation' or 'snipe_reservation' by focusing on cancellation rather than creation or automated acquisition. However, it doesn't explicitly differentiate from 'cancel_snipe' which might cancel automated reservations versus regular ones.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing reservation), exclusions (e.g., cannot cancel after a certain time), or comparisons with sibling tools like 'cancel_snipe' or 'list_reservations'. The agent must infer usage from context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• cancel_snipeC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. While 'Cancel' implies a destructive mutation, the description doesn't specify whether this action is reversible, what permissions are required, whether it affects other resources, or what happens on success/failure. It provides minimal behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's appropriately sized for a simple tool with one parameter and gets straight to the point without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive operation tool with no annotations and no output schema, the description is inadequate. It doesn't explain what constitutes a 'snipe attempt', what happens after cancellation, whether there are side effects, or what the tool returns. Given the complexity of cancellation operations and lack of structured safety information, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'snipe_id' well-documented in the schema. The description doesn't add any additional semantic context about the parameter beyond what the schema already states. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Cancel') and target resource ('a scheduled snipe attempt'), providing specific verb+resource pairing. However, it doesn't differentiate from sibling tools like 'cancel_reservation' or explain what distinguishes a 'snipe' from other operations in this context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'cancel_reservation' or 'list_snipes'. There's no mention of prerequisites (e.g., needing an active snipe), error conditions, or typical workflow context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_auth_statusC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it states the tool checks credential validity, it doesn't describe what 'valid' means (e.g., authentication status, token expiration), what happens if credentials are missing or invalid (e.g., error messages, return values), or any side effects (e.g., rate limits, logging). For a tool with no annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the tool's purpose without unnecessary words. It is front-loaded with the core action ('Check') and resource ('credentials'), making it easy to parse. Every part of the sentence earns its place, and there is no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that checks credential status. It doesn't explain what the tool returns (e.g., success/failure indicators, detailed validity information) or handle edge cases (e.g., what if no credentials are set). For a tool in a credential management context with siblings like set_credentials, more completeness is needed to guide the agent effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the parameter 'platform' fully documented in the schema (including enum values and default). The description adds no additional parameter semantics beyond what the schema provides, such as explaining the implications of choosing 'all' versus specific platforms. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check if credentials are configured and valid for each platform.' It specifies the verb ('check') and resource ('credentials'), and distinguishes it from credential-setting tools like set_credentials or set_login. However, it doesn't explicitly differentiate from other credential-related tools like refresh_token, which slightly reduces clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., whether credentials must be set first), nor does it suggest scenarios where this check is necessary (e.g., before making a reservation). With sibling tools like set_credentials and refresh_token, explicit usage context would help the agent decide when to invoke this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_availabilityC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool's function but doesn't describe traits like whether it's read-only (implied by 'Get'), potential rate limits, authentication requirements, error handling, or output format. For a tool with no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that front-loads the core purpose without unnecessary words. It efficiently communicates the tool's function, making it easy for an agent to parse and understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 required parameters, no output schema, and no annotations), the description is incomplete. It lacks details on behavioral traits, usage context, and output expectations, which are crucial for an agent to invoke it correctly. The high schema coverage helps, but the description doesn't compensate for other gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds no additional meaning beyond implying the parameters are used to fetch availability, which aligns with the schema but doesn't provide extra context like parameter interactions or examples. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get available time slots') and the resource ('for a specific restaurant on a given date'), making the purpose immediately understandable. However, it doesn't explicitly differentiate this tool from potential siblings like 'search_restaurants' or 'make_reservation', which might also involve availability-related functions, so it doesn't reach the highest score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., authentication status from 'check_auth_status'), exclusions (e.g., not for booking), or comparisons to siblings like 'search_restaurants' (which might list restaurants without checking slots). This lack of contextual direction leaves the agent to infer usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_reservationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'View your upcoming reservations' implies a read-only operation, but it doesn't specify authentication requirements, rate limits, pagination behavior, or what happens with no reservations. For a tool with zero annotation coverage, this minimal description leaves significant behavioral questions unanswered about how the tool actually works in practice.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just four words: 'View your upcoming reservations.' Every word earns its place - 'View' specifies the action, 'your' indicates ownership/authentication context, 'upcoming' provides temporal scope, and 'reservations' identifies the resource. There's zero waste or redundancy in this minimal but complete statement of purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple single-parameter tool, the description is incomplete. While it states the basic purpose, it doesn't address authentication requirements (implied by 'your' but not explicit), return format, error conditions, or how it differs from similar listing tools. For even a simple tool, more context about what 'view' actually returns would be helpful for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'platform' fully documented in the schema (enum values, default, description). The tool description adds no parameter information beyond what's already in the structured schema. According to scoring rules, when schema_description_coverage is high (>80%), the baseline is 3 even with no param info in the description, which applies here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with 'View your upcoming reservations' - a specific verb ('View') and resource ('upcoming reservations'). It distinguishes from siblings like 'make_reservation' (creation) and 'cancel_reservation' (deletion), though it doesn't explicitly differentiate from 'list_snipes' which might be similar in listing functionality. The purpose is clear but could be more specific about what distinguishes it from other listing tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With siblings like 'list_snipes' (likely listing different reservation types) and 'check_availability' (checking future availability rather than existing reservations), there's no indication of when this specific listing tool is appropriate versus other listing or checking tools. The description simply states what it does without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_tokenC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions 'manually refresh' but doesn't clarify if this is a read-only or mutating operation, what permissions are required, whether it affects existing sessions, or what happens on failure (e.g., token expiration). For a security-sensitive tool, this leaves critical behavioral traits undocumented.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('Manually refresh authentication token') and adds necessary detail ('using stored login credentials'). There is no wasted verbiage or redundant information, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of authentication operations and the lack of both annotations and an output schema, the description is insufficient. It doesn't explain what the refreshed token is used for, how it's returned (e.g., as a string or in a response object), or error conditions. For a tool with security implications, this leaves too many contextual gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'platform' fully documented in the schema (including enum values like 'resy'). The description adds no parameter-specific information beyond what the schema provides, such as explaining why 'platform' is needed or how it influences the refresh process. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('refresh') and target ('authentication token'), and specifies the method ('using stored login credentials'). It distinguishes itself from authentication-related siblings like 'check_auth_status', 'set_credentials', and 'set_login' by focusing on token renewal rather than status checking or credential setting. However, it doesn't explicitly contrast with all siblings, keeping it at a 4 rather than a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'check_auth_status' (for verifying token validity) or 'set_credentials'/'set_login' (for initial credential setup). It lacks explicit when-to-use or when-not-to-use instructions, leaving the agent to infer usage from context alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_snipesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a 'View' operation, implying read-only behavior, but doesn't specify if it requires authentication, returns real-time or cached data, includes pagination, or has rate limits. For a tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('View all scheduled snipe attempts and their status') with zero wasted words. It's appropriately sized for a no-parameter tool and gets straight to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters and no output schema, the description provides a basic understanding of its purpose. However, with no annotations and sibling tools present, it lacks context on authentication needs, data format, or differentiation from alternatives, making it only minimally adequate for a simple read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema description coverage is 100%, so there's no need for parameter documentation in the description. The description correctly avoids discussing parameters, earning a high baseline score for not adding unnecessary information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('View') and resource ('all scheduled snipe attempts and their status'), making the purpose immediately understandable. However, it doesn't explicitly differentiate this tool from sibling tools like 'list_reservations' or 'snipe_reservation', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With sibling tools like 'list_reservations' and 'snipe_reservation' available, there's no indication of whether this tool is for monitoring active snipes, historical attempts, or something else, leaving the agent to guess.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_credentialsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions 'securely store' which implies a write operation, it doesn't disclose important behavioral traits like whether this requires admin privileges on Windows, whether credentials are encrypted, what happens if credentials already exist, or if there are rate limits. The description is too sparse for a security-sensitive tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point with zero wasted words. It's appropriately sized for a tool with clear parameters and no complex behavioral nuances needing explanation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a security-sensitive credential storage tool with no annotations and no output schema, the description is inadequate. It doesn't explain what happens after storage (success/failure indicators), whether the tool validates credentials, or how stored credentials relate to other tools like 'check_auth_status'. The description should provide more context about this critical security operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the schema already documents all three parameters. The description adds no additional parameter semantics beyond what's in the schema - it doesn't explain the relationship between api_key and auth_token, when each is required, or format requirements. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('securely store'), the resource ('API credentials for Resy or OpenTable'), and the location ('Windows Credential Manager'). It distinguishes this tool from sibling tools like 'set_login' by specifying it's for API credentials rather than general login credentials.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'set_login' or 'refresh_token'. It doesn't mention prerequisites, timing considerations, or exclusions for when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_loginA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses key behavioral traits: it stores credentials for token refresh and encrypts them in Windows Credential Manager, which is useful context. However, it lacks details on permissions, security implications, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and efficiently adds encryption details in a second sentence, with zero wasted words. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides basic purpose and storage context but lacks completeness for a security-sensitive tool. It misses details on success/failure responses, validation, or integration with other tools like 'refresh_token'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters. The description adds no additional meaning beyond what the schema provides, such as format or usage details for the parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Store email/password for automatic token refresh') and resource ('Credentials are encrypted in Windows Credential Manager'), distinguishing it from sibling tools like 'set_credentials' by specifying the storage mechanism and purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives like 'set_credentials' or 'refresh_token' is provided. The description implies usage for token refresh but lacks context on prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• make_reservationA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses key behavioral traits: platform-dependent outcomes (completes booking vs. returns URL) and dependency on 'check_availability' for slot_id. However, it doesn't cover permissions, error handling, or mutation effects, leaving gaps for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste: first states purpose, second adds critical platform-specific behavior. Front-loaded with essential information, no redundant details. Every sentence earns its place efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a 5-parameter write tool. It covers purpose and platform behaviors but lacks details on return values, error cases, or full behavioral context. Adequate as a minimum viable description but with clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description adds minimal value beyond schema: it hints at slot_id's source ('from check_availability') and platform behaviors, but doesn't explain parameter interactions or semantics. Baseline 3 is appropriate as schema does heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Book') and resource ('reservation at a restaurant'), and distinguishes between two platform behaviors (Resy completes booking, OpenTable returns URL). This is specific and differentiates it from siblings like 'check_availability' or 'cancel_reservation'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning platform-specific behaviors, but doesn't explicitly state when to use this tool versus alternatives like 'snipe_reservation' or prerequisites. It hints at context with 'from check_availability' for slot_id, but lacks clear when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• snipe_reservationA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions automation and timing ('automatic booking attempt for the exact moment slots become available'), but lacks details on permissions, rate limits, success/failure behavior, or what happens after scheduling (e.g., confirmation, notifications). For a tool with 6 parameters and no annotations, this is a significant gap in transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero waste, front-loaded with the core purpose and followed by context. Every word earns its place, making it highly concise and well-structured for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, no annotations, no output schema), the description is adequate but incomplete. It covers the purpose and usage context well, but lacks behavioral details (e.g., what the tool returns, error handling) and does not compensate for the absence of annotations or output schema, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description does not add any parameter-specific semantics beyond what the schema provides (e.g., it doesn't explain interactions between parameters like 'release_time' and 'preferred_times'). Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Schedule an automatic booking attempt') and resource ('slots'), and distinguishes it from siblings like 'make_reservation' by emphasizing automation for time-sensitive releases. It explicitly mentions the target scenario ('popular restaurants that release reservations at specific times'), making the purpose highly specific and differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool ('for popular restaurants that release reservations at specific times'), which implicitly guides usage versus alternatives like 'make_reservation' for immediate bookings. However, it does not explicitly state when not to use it or name specific alternatives, missing full explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_restaurantsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful context beyond basic functionality, such as the default platform search behavior ('searches BOTH Resy and OpenTable by default') and what information is included in results ('ratings, cuisine type, and price range'). However, it lacks details on rate limits, error handling, or pagination, which are important for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, with two sentences that efficiently convey the tool's purpose, scope, and output. Every sentence adds value without redundancy, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (6 parameters, no output schema, no annotations), the description is adequate but has gaps. It covers the search functionality and result fields well, but lacks details on behavioral aspects like rate limits or error handling. Without an output schema, it should ideally explain return values more explicitly, though it does mention key fields like 'platform'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds minimal value beyond the schema, mentioning searchable fields ('by name, cuisine, or location') and the default platform behavior, but does not provide additional syntax or format details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('search for restaurants') and resources ('restaurants'), and distinguishes it from siblings by focusing on search rather than reservation management. It specifies the search scope (name, cuisine, location) and platforms covered (Resy and OpenTable), making it highly specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('search for restaurants by name, cuisine, or location') and mentions the default behavior (searches both platforms). However, it does not explicitly state when not to use it or name alternatives among sibling tools (e.g., check_availability for specific booking queries), which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.