Server Quality Checklist

Tool Scores

• opentable_get_restaurantB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It mentions the output includes description, address, hours, and features, but it doesn't disclose whether the tool requires authentication (likely, given sibling tools include login/logout), whether it's a read-only operation, or any rate limits or potential errors. The description is too thin on behavioral context beyond the basic data fields.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence, front-loaded with the core purpose and immediately listing what's included. No fluff, easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple (one param, no output schema), but with no annotations, the description should compensate. It fails to mention authentication requirements (given sibling tools for login/logout), whether the operation is read-only, or if the restaurantId might be a URL vs an ID, which could affect invocation. The description covers the basics but lacks critical contextual details for a real-world API tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter, restaurantId, is fully documented in the schema (says it's the restaurant ID or profile URL from search results). The description adds no extra parameter-related meaning beyond confirming the scope. Schema coverage is 100%, so baseline is 3; no additional value provided by the description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves detailed information about a restaurant, listing specific data types (description, address, hours, features). It distinguishes itself from sibling tools like opentable_search (which likely returns summaries) by focusing on detailed information for a specific restaurant, though it doesn't explicitly name the alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it should be used when detailed info is needed after identifying a restaurant via search (since restaurantId comes 'from search results'). It doesn't explicitly say when not to use it, but the context is clear: use this for specific restaurant details, not for listing/searching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_searchB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does not mention that this is a read-only operation, any potential rate limits, external API dependencies, or whether authentication is needed. The bare statement 'Search for restaurants' implies a read action but fails to disclose important behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that is front-loaded with the verb and resource. It packs essential information without any filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the rich schema (100% parameter coverage), the description adequately conveys the core functionality. It does not describe the output format or result ordering, but the action 'Search for restaurants' implies returning a list. It also does not explicitly differentiate from availability checking, which might be a minor gap, but overall it is reasonably complete for a search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description lists the parameters in prose (location, cuisine, party size, date, time) but adds no additional meaning beyond what is already in the schema. It does not clarify required vs optional fields or provide default values, which the schema already handles.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Search') and the resource ('restaurants on OpenTable'), and lists the specific filter dimensions (location, cuisine, party size, date, time). This distinguishes it from sibling tools like opentable_check_availability (which likely checks specific restaurant availability) and opentable_get_restaurant (which fetches details).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives. It does not mention any exclusions or prerequisites, such as whether login is required or whether this should precede availability checks. The description only states what it does, leaving the AI to infer usage context from sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_check_availabilityA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It conveys a read-only intent via 'check,' but it does not explicitly state that no reservation is created, or describe auth, error, or empty-result behavior. It is minimally adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler, front-loaded with the action. Every word contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The action and required parameters are adequately documented, but there is no output schema and the description does not clarify the return shape or how this step relates to making or viewing reservations. Adequate for a simple lookup but leaves some context implicit.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no parameter-specific detail beyond the schema; it only reiterates date and party size as scoping context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('check') and resource ('available reservation time slots') with clear scope: restaurant, date, and party size. It clearly distinguishes this from sibling tools like opentable_make_reservation and opentable_get_reservations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance on when to use this tool versus alternatives, such as using opentable_get_reservations to view existing bookings or opentable_make_reservation after selecting a slot. Usage is only implied by the verb 'check'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_make_reservationA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it delivers by revealing that the user must be logged in and by explaining the preview-vs-commit workflow. However, it doesn't discuss potential side effects like double-booking or idempotency of repeated calls, though the confirm mechanism mitigates concern.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, with each one serving a distinct purpose: stating the core action, explaining the critical delete-style safety mechanism, and flagging a prerequisite. Every word earns its place, and no information is repeated or redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters and no output schema, the description covers the key operational aspects: the preview/booking flow and authentication requirement. The main gap is silence on what the tool returns after a successful booking (e.g., confirmation details), which is left undefined since no output schema exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description reiterates the confirm parameter logic, which reinforces its importance but adds nothing new beyond the existing schema descriptions. It confirms the confirm-flow but doesn't inform on specialRequests, date, or other params in a way that goes beyond the schema's documented format requirements.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource construction ('Book a restaurant reservation on OpenTable') and the additional context about confirm=true/false clarifies the dual preview/booking behavior. This clearly differentiates it from siblings like check_availability, get_reservations, and cancel_reservation, which serve distinct purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a usage pattern ('Set confirm=false to preview before booking') but never explicitly states when to use this tool versus alternatives like opentable_check_availability or opentable_get_reservations. The workflow guidance is implied through the confirm parameter rather than stated as explicit tool-selection criteria, which puts it at the 'implied usage' baseline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_cancel_reservationA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full responsibility for behavioral transparency. It explicitly warns 'This action cannot be undone' and explains the preview vs. actual cancel behavior, which are important side-effect disclosures.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, each with a distinct and necessary purpose: stating the action, explaining the confirm flag, and warning about irreversibility. No extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter cancellation tool with no output schema or annotations, the description covers the critical aspects: the operation, the preview workflow, and the irreversible consequence. It does not describe the return value or success behavior, but this is not essential for this simple action.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents both confirm and reservationId. The description also repeats the confirm semantics but adds no meaning beyond the schema, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states exactly what the tool does: 'Cancel an existing reservation.' This is a specific verb and resource that clearly distinguishes the tool from siblings like opentable_make_reservation or opentable_get_reservations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear guidance on how to use the confirm parameter: 'Set confirm=false to preview, confirm=true to actually cancel.' This gives practical usage context, though it does not explicitly discuss when to choose this over alternatives (which are not applicable here).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_get_reservationsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that authentication is required, which is a key behavioral trait. However, it does not mention anything about the return format, ordering, whether it's read-only (implicit but not stated), or any potential errors. For a simple list operation, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, with the primary purpose front-loaded and the authentication note as a secondary requirement. No filler or redundancies; every word is informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although there is no output schema, the tool's simplicity (no params, unambiguous purpose) means the description covers the essential details: what it lists, for whom, and the auth prerequisite. It could mention the shape of the response, but given the straightforward nature and sibling context, it is mostly complete. A slight gap is lack of explicit mention of read-only behavior, but that is inferred from 'list'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is 100% (vacuously). Per the rubric, a 0-parameter tool receives a baseline of 4. The description adds context about its scope ('upcoming') which is useful, but there is nothing further needed since there are no parameters to elaborate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'upcoming reservations' for the logged-in user, which is specific and distinguishable from siblings like search or get_restaurant. It also implies a read-only action, aligning with the plural reservations vs. single restaurant focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly notes the authentication requirement, which is a clear prerequisite. While it doesn't mention when not to use it or name alternatives, the description's directness and the sibling set make it obvious this is for retrieving reservations, not for searching or booking. The context is clear but lacks explicit exclusion guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_loginA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It tells what the tool does (gets URL/instructions) and implies that the user performs authentication, but does not disclose any side effects, prerequisites, or expected return structure. This is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose and then the usage condition. Every word earns its place—no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema tool, the description is sufficiently complete. It names the exact artifact (login URL and instructions) and when to use it, leaving little ambiguity. It could mention whether it returns anything in a specific format, but the description's clarity compensates for that as a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so schema coverage is vacuously 100%. The description adds no parameter information because none is needed. A baseline of 4 is appropriate since there is nothing to clarify.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gets the OpenTable login URL and instructions for the user to authenticate, which is a specific verb-resource pair. It distinguishes itself from siblings by focusing on login and explicitly referencing opentable_status as a precursor condition.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides an explicit when-to-use condition ('Use this when opentable_status returns not logged in'), which gives clear usage context. It does not mention exclusions or alternatives, but the conditional is sufficient and directly connects to a sibling tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_logoutA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavior. It does state the core effect (clearing cookies, resetting auth state) but omits potential side effects like whether it invalidates server-side sessions, makes network calls, or is idempotent. This is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no redundant words. It front-loads the action ('Clear the stored OpenTable session cookies') and adds a brief usage note, all in under 30 words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter logout tool with no output schema, the description covers the essential purpose and usage. It could elaborate on edge cases (e.g., behavior when already logged out) but is sufficiently complete for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100%. Per the rubric, a baseline of 4 is appropriate; the description adds no parameter-specific details but none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Clear the stored OpenTable session cookies' and explicitly equates it to 'log out or reset the authentication state.' This is a specific verb-resource pairing that distinguishes it from siblings like opentable_login and opentable_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The statement 'Use this to log out or reset the authentication state' provides clear guidance on when to use the tool. Though it doesn't mention specific alternatives, the purpose is unambiguous and aligns with the sibling context (e.g., login handles the opposite action).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opentable_statusA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the tool returns login status and instructions if not authenticated, indicating a read-only check with conditional output. However, it doesn't explicitly mention that it has no side effects or that it doesn't perform authentication itself, though that is implied. It provides useful context about the return value.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: it states the purpose, the return value, and the usage context in two sentences. No unnecessary words, and it effectively communicates the essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple status check with no parameterslint: It explains what it does and when to call itborn. However, it doesn't specify the exact structure of the 'login status' or what 'instructions' might include, but given the simplicity, it's reasonably complete. A 4 is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters)Skip

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: checking if the user is logged in to OpenTable. It specifies a direct action (check) on a specific resource (login status), and distinguishes itself from sibling login/logout tools by focusing on status rather than performing the action. It also mentions the return of login status and instructions, which clarifies the primary output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs to call this before any other OpenTable operations, giving clear when-to-use guidance. It implies it should be the first step, which differentiates it from the other tools and sets a precondition. Though it doesn't explicitly say when not to use it, the directive 'before any other' is sufficiently explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.