Server Quality Checklist

Tool Scores

• check_inventoryA
  4.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral transparency. It does not explicitly state that the operation is read-only, nor does it mention authentication, rate limits, or other side effects. The read-only nature is implied but not disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences, front-loading the core purpose and then giving a practical usage guideline. There is no unnecessary detail or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description compensates by explaining the output shape: stock broken down by location with available, committed, and on-hand counts. It also provides the decision context, making it complete for this simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides descriptions for both parameters ('Exact SKU to look up' and 'Maximum matching variants, 1-50'), so schema coverage is high. The tool description adds context about the response structure but does not further clarify parameter semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns stock levels for a SKU, broken down by location with available, committed, and on-hand counts. It also distinguishes this from product-level queries by noting the 'can we ship it' use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly tells when to use this tool: when the question is about shippable availability ('can we ship it') rather than whether the item is sold ('do we sell it'). This provides clear guidance relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_productsA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden for behavioral transparency. It states that only identity and total inventory are returned, but it does not explicitly mention whether the operation is read-only or if there are other side effects, auth requirements, or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, using three focused sentences. It front-loads the primary purpose, then clarifies output limitations and usage context without unnecessary detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and the description is sufficient for invocation, including query syntax and response scope. However, no output schema is provided and the description only names 'identity and total inventory' without specifying exact fields, leaving some minor ambiguity about the return shape.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful detail to both parameters: query is explained with Shopify search syntax examples, and limit's range is reiterated in the description. Schema coverage is 100%, and the description enhances understanding of how to construct effective queries.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: searching/finding products by free text, vendor, or status. It distinguishes this from get_product by noting that this is the entry point when the handle is unknown.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to use this tool first when you do not already know a handle, and directs users to get_product for per-variant detail. This provides clear when-to-use and alternative guidance relative to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• low_stock_reportA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of explaining behavior. It discloses that the tool 'Scans up to `scan` variants and filters locally,' including a performance tip to 'raise `scan` for a large catalogue.' This gives agents important context about potential limitations and how to adjust parameters accordingly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is just two sentences, yet conveys purpose, usage, parameter behavior, and a practical tip. It is tightly worded without redundancy, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and simple parameters, the description is essentially complete. It explains what is returned (variants at or below threshold), the sort order, and the parameter effects. A minor gap is not explicitly stating the output format, but that is not critical for this kind of report.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already fully covers both parameters with descriptions. The tool description adds further meaning by explaining the interaction: the threshold determines what qualifies as low stock, and the scan parameter controls how many variants are examined. This enriches the schema's static explanations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Variants at or below a stock threshold, lowest first.' This is a specific action (reporting) on a specific resource (variants). It distinguishes itself from sibling tools like search_products and check_inventory by focusing on low-stock items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use for restock questions,' providing direct guidance on when to invoke this tool over alternatives. Though it doesn't name sibling tools, the use case is clearly differentiated from search/get/inventory operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description fully carries the burden of explaining behavior. It clearly indicates this is a read operation ('Read one product in full'), describes the extent of data returned (variants with SKU, price, stock level), and does not imply any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, using two sentences to convey the tool's purpose, the data returned, and the parameter constraint. No unnecessary words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple fetch operation without an output schema, the description provides sufficient context: what it does, what data is included, and how to specify the target. It does not leave critical gaps that would prevent correct usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful constraints beyond the schema by stating that exactly one of the two parameters (id or handle) must be used. This clarifies the relationship and use of the parameters, which the schema alone does not fully convey.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a single product in full, including variants with SKU, price, and stock level. It distinguishes itself from siblings like search_products and check_inventory by focusing on retrieving a specific product's complete data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs that the product must be identified by handle or by gid://shopify/Product/... id and that exactly one of the two must be provided. This gives clear guidance on when to use this tool versus alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.