Server Quality Checklist

Tool Scores

• search_productsC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states that the tool performs a search; it does not mention whether it is read-only, how it handles pagination (e.g., 'limit'/'offset' parameters), or any side effects. The phrase 'objective product and ingredient filters' hints at query capabilities but lacks specifics about return behavior or constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence, but it is under-specified for a tool with 14 parameters and no other documentation. While there is no fluff, the extreme brevity leaves out essential context, making it only partially 'appropriately sized.'

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (14 parameters, no annotations, no schema description coverage), the description is insufficient. It provides a high-level purpose but does not explain filtering capabilities, result format, or usage context. An output schema exists, but the description alone is too thin for an agent to understand the tool's full scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 14 parameters with zero description coverage, and the tool description does not compensate. It only generically refers to 'product and ingredient filters,' which does not explain any specific parameter (e.g., 'upc', 'brands', 'include_ingredients'). The description adds no meaning beyond the raw parameter names and types in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Search supplement labels using objective product and ingredient filters.' The verb 'Search' is specific to the resource 'supplement labels,' and the mention of filters differentiates it from sibling tools like 'get_product' or 'compare_products.' This is a clear and distinguishing purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'get_product' or 'compare_products.' There are no explicit conditions, exclusions, or references to sibling tools, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It mentions 'complete' and 'normalized,' which are behavioral traits, but does not disclose what happens if the product is not found, whether the operation is read-only, or any potential side effects. The description is too minimal to inform the agent of safety or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with the verb front-loaded. It is concise, clear, and contains no fluff or repetition. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple one-parameter getter with an output schema, the description is minimally viable. It states the core action and object, and the output schema covers return values. However, it lacks usage context, especially regarding sibling tools, and provides no parameter details, leaving gaps for an agent to select it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it does not explain 'product_id' at all. The parameter name is self-explanatory only by convention; no format, usage, or semantics are provided. The description adds no value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (Return) and the resource (one complete normalized supplement label record), which distinguishes it from siblings like search_products or compare_products. It specifies the scope (single record) and the type of data (supplement label), making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention that this is for fetching a specific product by ID, nor does it point to search_products for discovery or compare_products for comparison. There is no explicit when/when-not or alternative naming.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• calculate_stackB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full transparency burden. It only states the high-level operation without disclosing side effects, error handling, read-only status, or the computation logic. The phrase 'Sum' implies a read operation, but key behavioral details are absent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundant words, front-loading the action verb 'Sum'. It is appropriately sized for a simple tool, though brevity limits explanatory depth.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite having an output schema and a nested input structure, the description omits important context such as return value, usage scenarios, and integration with sibling tools. It fails to mention that multiple product selections can be summed, leaving the tool's full capabilities unclear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It vaguely mentions 'product and serving combination' but does not clarify the roles of product_id or servings_per_day, nor that selections is an array allowing multiple entries. This adds minimal value beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action ('Sum') and resource ('known label amounts') for a specific context (product and serving combination). It distinguishes itself from sibling tools like search_products and get_product by using a calculation-oriented verb, avoiding tautology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for totaling label amounts, but provides no explicit guidance on when to use this tool versus alternatives. It does not mention sibling tools or any exclusions, leaving usage context largely inferred.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_productsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It implies a read-only comparison operation and adds 'per-serving label facts' context, but does not explicitly disclose side effects, data sources, or behavior beyond the basic action. Since it is clearly a non-destructive comparison, a 3 is appropriate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It states the verb, resource, and key constraint efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema present), the description covers the essential purpose and the minimum product count. No critical details are missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does by clarifying that the product_ids array must contain at least two product identifiers ('two or more products'), which is a critical constraint not visible in the schema. This adds meaningful semantic value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (compare), the resource (products), and the specific scope (per-serving label facts). It naturally distinguishes from siblings like get_product (single product) and search_products (finding products).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when comparing multiple products but provides no explicit guidance on when to use this tool versus alternatives. Sibling tools exist and could be called out, but the verb 'compare' gives some contextual cue.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_catalog_infoA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It clearly indicates a read-only operation (via 'Return') and specifies the outputs, which covers the key behavioral trait. Minor details like potential performance implications are absent, but given the tool's simplicity, the description is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence states the verb and outputs directly. No wasted words; every element contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only tool with an output schema (which presumably documents the return structure), the description is sufficiently complete. It lists all key outputs (catalog version, product count, snapshot metadata) and requires no further explanation for invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema coverage is trivially 100%. The baseline for 0 parameters is 4, and the description appropriately adds no parameter-specific semantics since none are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Return catalog version, product count, and snapshot metadata' uses a specific verb and lists concrete resources, making the tool's purpose unmistakable. It clearly distinguishes itself from siblings like search_products and get_product, which focus on individual product queries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or alternative guidance is provided. The intended use (when catalog-level metadata is needed) is implied by the description and tool name, but unlike the TDQS high example, it does not explicitly state exclusions or recommend alternative tools for other scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.