Server Quality Checklist

Tool Scores

• set_delivery_slotA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'Select a delivery slot' and does not state whether this mutates the current order/cart, replaces an existing slot selection, requires authentication/2FA, or has other side effects. This is a meaningful gap for a setter tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler: it states the action, the input, and the relevant sibling reference. It is front-loaded and appropriately sized for a one-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter and an output schema present, the description does not need to explain return values. But as a mutation tool with no annotations, it lacks context about side effects and what the selected slot applies to (e.g., current cart/order), leaving the agent to infer the workflow from siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description must compensate, and it does identify slot_id as the delivery-slot identifier and directs the agent to get_delivery_slots for valid values. However, it offers no format, constraints, or behavior if the slot is invalid or expired, so it only partially compensates for the schema's lack of documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Select') and resource ('delivery slot'), and identifies the exact input needed (slot_id). It also points to the sibling get_delivery_slots, distinguishing this mutation/set operation from its lookup counterpart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The parenthetical 'see get_delivery_slots' explicitly directs the agent to the sibling tool that supplies the slot_id, providing clear prerequisite context. It does not enumerate when-not-to-use cases, but no other sibling directly competes with this setter, so the guidance is adequate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_2fa_codeA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the operation and prerequisite but does not describe what happens on success or failure, whether the code is single-use or expires, or what side effects completing login has. This leaves important behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler. It front-loads the main action and immediately supplies the relevant context about where the code comes from. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter tool with an output schema, the description is nearly complete: it states the purpose, the prerequisite, and where the parameter value comes from. It does not explain failure behavior or edge cases, but an agent can select and invoke the tool correctly with the given context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It does identify 'code' as the 2FA code generated by generate_2fa_code, which is helpful, but it gives no information about expected format, length, or how errors occur. The schema itself only provides the parameter name and type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names the specific operation ('verifying the 2FA code') and the resource ('2FA code'), and ties it to the related generate_2fa_code tool. It clearly distinguishes itself from siblings like generate_2fa_code because this verifies rather than generates.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys the intended flow: the code must have been requested via generate_2fa_code, and this call completes login. It does not explicitly state when not to use it or name alternatives, but the prerequisite relationship is clear enough for an agent to sequence calls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_to_cartA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the behavioral burden. It discloses the return behavior ('Returns the updated cart, same shape as get_cart()'), which is useful. It does not mention error handling, persistence side effects, or quantity constraints, but for a simple cart mutation the disclosed behavior is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The main action and parameter guidance come first, and the return-shape note follows efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter tool with no output schema and no annotations, the description covers the key inputs, the source of product_id, and the return shape via the get_cart reference. It does not discuss edge cases, but the information needed for normal invocation is largely present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must clarify parameters. It explains that product_id comes from search_products and that count represents units of the product. This adds meaning beyond the bare schema types and names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action: 'Add `count` units of a product to the cart', which is a specific verb and resource. It also references product_id from search_products, distinguishing this addition tool from siblings like remove_from_cart and clear_cart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used after search_products by stating 'product_id from search_products', and the return-shape note connects it to get_cart. However, it does not explicitly explain when to use this tool versus alternatives such as remove_from_cart, clear_cart, or get_cart.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_cartA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It clearly states the return payload and that the entire basket is returned, but it does not explicitly confirm that the operation has no side effects. The word 'Get' strongly implies read-only behavior, but this is not spelled out.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The core purpose is front-loaded, and the return fields are listed compactly in a structured way.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read tool with no output schema, the description sufficiently covers what the tool does and what it returns. It does not mention edge cases like an empty cart or pricing currency, but those are minor given the simplicity of the tool and the clear sibling context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema already provides full coverage of that fact. The description adds no parameter details because none are needed, matching the baseline for a no-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Get the full Picnic basket') and details the returned fields, making it instantly clear this is a read operation for the entire cart. It is distinct from sibling cart mutation tools such as add_to_cart, remove_from_cart, and clear_cart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is used to view the complete cart, but it does not explicitly state when to choose it over alternatives or when not to use it. There are no exclusions or conditions, so the guidance is adequate but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_delivery_slotsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the behavioral transparency burden. It clearly indicates a read-only listing operation and details the fields returned, including the meaningful `suggested` flag. However, it does not disclose edge cases such as behavior with an empty or missing cart, or confirm that no state changes occur.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core action and scope, then efficiently lists the returned fields in a readable format. There is no filler or redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and an output schema exists, the description provides sufficient context for correct invocation: it identifies the scope ('current cart'), the action (list slots), and the key fields, including the `suggested` default flag. Minor additional context about empty-cart behavior or required cart state could improve completeness, but it is not a significant gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter-level explanation is unnecessary. The description references the implicit 'current cart' context, which is the only relevant input, and that is handled clearly. Baseline 4 is appropriate for a no-parameter tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') and a clear resource ('available delivery slots for the current cart'), making the tool's purpose immediately obvious. It also naturally differentiates from the sibling set_delivery_slot, since this tool is about reading slots, not modifying them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool — when you need to see available slots for the current cart — but it does not explicitly state when not to use it or name alternatives like set_delivery_slot. This leaves usage guidance somewhat implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remove_from_cartA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does disclose the mutation (removing units) and the return behavior (updated cart shaped like get_cart()). However, it does not mention edge cases such as what happens when count exceeds the cart quantity or when the product is absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief, front-loaded with the primary action, and the second sentence adds useful return-shape information without fluff. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter cart operation, the description provides the essential context: what to remove, where product_id comes from, and the return shape. It lacks edge-case/error details, but the tool is low-complexity and siblings provide surrounding context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must add meaning to both parameters. It does clarify that count represents units to remove and that product_id comes from get_cart/search_products, adding value beyond the schema's type/default information. It could also specify that count should be a positive integer.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Remove count units of a product from the cart') and identifies the resource. It also tells the agent where to obtain a valid product_id, which separates it from sibling tools like add_to_cart and clear_cart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for use by pointing to get_cart/search_products as the source for product_id and explaining the return shape. It does not explicitly mention when not to use it or compare it against clear_cart/add_to_cart, but the intended usage is evident.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• clear_cartA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full disclosure burden. It clearly states the destructive effect (emptying the cart entirely) and the resulting state (returns the now-empty cart). This is sufficient for a simple zero-parameter mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, focused sentence that front-loads the action and result with no filler. Every word contributes to understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-annotation tool with no output schema, the description fully covers what the tool does and what it returns. There are no missing details that an agent would need to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so baseline is 4. The description does not need to explain parameter meaning, and it adds useful context about the command's effect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Empty') with a clear resource ('the Picnic cart') and specifies the scope ('entirely'), which distinguishes it from sibling tools like remove_from_cart that remove individual items. It also states the return value, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies when to use the tool: when the entire cart needs to be emptied. It does not explicitly name alternatives such as remove_from_cart or state when not to use it, but the word 'entirely' provides enough context for correct selection among cart-related siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_productsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It transparently describes the return structure, including id, name, price as a float in EUR, currency, and unit. It omits pagination or edge-case behavior, but for a simple search tool this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. It front-loads the action and resource, then immediately states the return format. This is exemplary conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter search tool with an output schema, the description provides everything necessary: what is searched, what is returned, and the type/currency of the price field. There are no notable gaps that would cause incorrect invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description is needed to explain the query parameter. The phrase 'Search Picnic's product catalog' clearly indicates that the query is the product search term, and the single parameter is self-explanatory; no further detail is essential.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Search' with a clear resource, 'Picnic's product catalog', and goes on to specify the return fields. This makes it unmistakably distinct from the sibling tools, which all concern cart, delivery, or 2FA operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is for finding products in the catalog, and no sibling tool overlaps with that function. It does not explicitly state when not to use it, but the context is sufficient given the unrelated sibling set.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_2fa_codeA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining the side effect: it sends a 2FA code via SMS or EMAIL. It also explains the account-requirement and the failure-driven use context. It does not mention rate limits, cooldowns, or that the code is delivered out-of-band, but the intended behavior is still clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three short sentences, each essential: the main purpose, the channel values, and the required workflow with the next tool. It is front-loaded with the action and is appropriately sized for a one-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and an output schema, the description covers the triggering condition, the parameter values, and the follow-up step. Nothing critical is missing for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It states that channel is 'SMS' or 'EMAIL', which gives the allowed values that the schema omits. It could note the default 'SMS' or explain the meaning of each channel, but the core parameter semantics are provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: 'Ask Picnic to send a 2FA code' for accounts that require it. It clearly differentiates from the sibling verify_2fa_code by focusing on sending/generating rather than verifying.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives an explicit trigger: 'Call this first if any other tool fails with a message about 2FA being required.' It also names the follow-up tool, verify_2fa_code, so the agent knows exactly when and in what sequence to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.