Server Quality Checklist

Tool Scores

• check_order_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It only mentions the authentication requirement. It does not explicitly state that this is a read-only operation, what happens if the order is not found, whether it returns historical orders when order_id is omitted, or any rate limits. This is minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences: 'Check the status of a Chick-fil-A mobile order. Requires authentication.' Every word earns its place; there is no redundancy or fluff. It is front-loaded with the core purpose and follows with a necessary prerequisite. Excellent conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description is adequate but leaves some gaps. It does not describe the return value or behavior when order_id is omitted, though the schema covers that. It also doesn't mention error scenarios. Given the tool's simplicity, a score of 3 reflects that it is minimally sufficient but not richly informative.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description covers 100% of the single parameter (order_id), including its optionality and the behavior of omitting it. The tool description adds no additional parameter information, so the baseline of 3 applies. The schema itself is quite clear, so the lack of description-level param detail is acceptable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Check the status of a Chick-fil-A mobile order.' This distinguishes it from siblings like 'place_order' and 'view_rewards' by focusing specifically on status checking. It could be a 5 if it explicitly named alternatives, but the verb+resource combination is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'place_order' or 'view_rewards'. The only contextual hint is 'Requires authentication,' which is a prerequisite rather than a usage directive. There is no mention of scenarios like 'after placing an order' or 'when you need a quick status update.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_nutritionB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries the full burden of behavioral disclosure. It does not state whether authentication is needed, what exactly 'detailed nutrition information' includes, what happens if the item is not found, or any side effects. The word 'Get' implies a read operation, but no further behavioral context is provided.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no unnecessary words. It is front-loaded with the action and resource, making it easy to understand quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, the description should explain what 'detailed nutrition information' returns and any failure modes. It does not. The single-parameter schema is well covered, but the tool is underspecified for a complete understanding of invocation and expected result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% for the single parameter item_name, with a clear description and examples. The tool description adds no additional meaning beyond the schema, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Get') and resource ('detailed nutrition information for a specific Chick-fil-A menu item'). It is specific and distinct from sibling tools like search_menu, which would focus on menu items rather than nutrition details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied: one would use this tool when needing nutrition information for a menu item. However, there is no explicit guidance on when to use it versus alternatives, nor any mention of exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_to_cartA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses the login requirement, but omits side effects on the cart, error behavior, or any success/failure response. This is minimal behavioral disclosure for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the action and followed by the key requirement. Every sentence earns its place with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The schema fully documents parameters, but the description lacks workflow context such as how to obtain valid item names (e.g., via search_menu) or what happens after adding to the cart. It is adequate for a simple action but leaves clear gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  All three parameters (item_name, quantity, customizations) have descriptions in the schema (100% coverage), so the description does not need to add parameter detail. It does not add any extra semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action 'Add a menu item to the Chick-fil-A mobile order cart' with a clear resource (menu item) and destination (cart). This distinguishes it from siblings like search_menu, place_order, and find_locations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a prerequisite ('Requires being logged in') but does not explicitly say when to use this vs alternatives such as search_menu or place_order. Usage context is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• place_orderA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of explaining behavioral impact. It mentions authentication and cart items, but fails to disclose that this places a real financial order, charges the payment method, is likely irreversible, or what success/error responses look like. This is a significant gap for a consequential purchase action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that front-loads the core action ('Place a mobile order') and then adds two essential prerequisites. Every word earns its place with no fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that triggers a real financial transaction, the description is incomplete. It covers prerequisites but omits key context such as order confirmation behavior, potential failure modes (e.g., invalid payment), and whether the order can be cancelled. The absence of an output schema further increases the need for description-led context, which is not met.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for 100% of parameters, so the description adds little beyond what is structured. The baseline of 3 applies here; the description does not provide additional syntax or nuance for parameters beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Place') and resource ('a mobile order at a Chick-fil-A location'), which clearly distinguishes it from sibling tools like add_to_cart and check_order_status. It communicates the core action without requiring the agent to infer from the tool name alone.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists prerequisites ('Requires authentication and items in cart'), providing clear context for when this tool can be used. It does not explicitly name alternatives or exclusions, but the prerequisites implicitly guide the agent to call this after authentication and cart setup.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_locationsA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only restates the basic function and does not mention what 'nearby' entails (e.g., default radius), what the response format is, or how invalid inputs are handled. For a tool with no annotations, this lacks meaningful behavioral insight.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that directly states the tool's purpose. There is no wasted text, and it is appropriately concise for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description and schema together are largely sufficient. However, without an output schema, the description does not clarify what the tool returns (e.g., a list of locations with addresses), which could be important for the agent. It is not severely incomplete, but a brief note on the return type would enhance completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters completely, so the baseline is 3. The description's mention of 'ZIP code or city name' aligns with the `zip_or_city` parameter and 'nearby' implies the radius, but it does not add information beyond what the schema already provides. Thus, it meets the baseline without improving parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Find nearby Chick-fil-A locations by ZIP code or city name.' It uses a specific verb ('Find'), a resource ('locations'), and search criteria ('ZIP code or city name'), distinguishing it from sibling tools that handle menus, nutrition, or orders.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool: whenever a user needs store locations based on a ZIP code or city. It does not explicitly exclude alternatives, but sibling tools are distinct enough that the intended use is clear. No further usage guidance or prerequisites are provided, which is acceptable for a simple lookup tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_menuA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden of behavioral disclosure. It does state the output ('Returns matching items with descriptions and prices'), which adds transparency. However, it omits details about search matching behavior (e.g., substring vs. exact match), handling of no results, or category filter semantics. This is minimal but acceptable for a simple search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no redundant information. The action and result are front-loaded, and every word contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple 2-parameter search tool with no output schema, the description adequately covers the purpose, input scope, and return content. It lacks explicit usage context (e.g., when to use during ordering), but the tool is straightforward enough that the information provided is sufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both query and category have descriptions). The description's phrase 'by name or keyword' merely reiterates what the schema already states for query. No additional parameter-level meaning is added, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Search' with the resource 'Chick-fil-A menu items' and specifies the search method ('by name or keyword'). It also states the return value ('matching items with descriptions and prices'), clearly distinguishing it from sibling tools like get_nutrition or find_locations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies the use case: searching for menu items to see descriptions and prices. However, it does not explicitly list when not to use it or mention alternatives (e.g., use get_nutrition for nutritional details). The context is clear but lacks explicit exclusions, so it earns a 4.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• view_rewardsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It reports the authentication requirement, which is useful, but does not explicitly confirm read-only behavior, rate limits, or other side effects. 'View' implies non-mutating, but more explicit transparency would earn a higher score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose, and contains no filler. Every word contributes to understanding the tool's function and requirement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple view tool with no parameters and no output schema, the description covers the essential aspects: what data is returned (rewards, points, tier, offers) and the authentication prerequisite. It leaves no significant gaps given the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially complete. The description correctly adds no parameter details since none exist, matching the baseline of 4 for parameterless tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('View') and resource ('Chick-fil-A One rewards, points balance, membership tier, and available offers'). It distinguishes itself from sibling tools like search_menu, place_order, and get_nutrition by focusing specifically on rewards and offers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by specifying that authentication is required and that this tool is for viewing rewards/offers. It implies when to use it (when rewards info is needed) but does not explicitly name alternatives or exclusion criteria, so it lacks a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.