Server Quality Checklist

Tool Scores

• get_customer_contactsD
  1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Tool has no description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Tool has no description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has no description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has no description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Tool has no description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Tool has no description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_customer_phoneD
  1/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Tool has no description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness1/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Tool has no description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool has no description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Tool has no description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose1/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Tool has no description.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Tool has no description.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• browse_customersB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must indicate side effects; it does so only by using the verb 'retrieves,' implying a read operation. It adds that results can be sorted/filtered and that customer_id returns a single comprehensive profile, but it does not describe pagination, returned fields, or any prerequisites or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded with the core purpose, followed by capabilities and the customer_id special case. There is minor redundancy between 'advanced filtering options' and 'filtering by minimum/maximum spend,' but no unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read tool with six optional parameters and no output schema, the description gives a reasonable overview but leaves gaps: the return format is only vaguely described as 'detailed customer profiles,' and no pagination or ordering default behavior is stated. The schema covers parameters, but tool selection would benefit from sibling-distinguishing context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all six parameters in detail. The description adds only general context (advanced filtering, min/max spend, comprehensive detail for customer_id) without introducing new parameter semantics beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('retrieves customer information'), the resource (customers), and notable capabilities ('spending data', 'advanced filtering options', 'sorting by multiple fields'). It does not explicitly differentiate from sibling tools like get_customer_contacts, but the mention of spending data and filtering narrows the scope clearly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use browse_customers versus alternatives such as get_customer_contacts or browse_orders. The only usage note, 'When provided with a specific customer_id, it returns comprehensive information for that single customer,' is parameter-level rather than tool-selection guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_product_variantB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the additive action and does not mention required permissions, error behavior (e.g., if the product is missing), idempotency, reversal effects, or what response is returned. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tightly written sentence communicates the core action and key attributes, with no filler or redundant phrases. Every part of the sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero annotations, no output schema, and a mutation operation, the description is too sparse. It does not address crucial behavioral context like what happens after creation, prerequisite checks, or failure modes. The schema covers parameters fully, but the operation context remains incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds no new parameter information beyond what the schema already documents (e.g., the exact order requirement for option_values is already in the schema). It only offers a high-level summary without deepening parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Adds'), a clear resource ('new variant to an existing product'), and the key attributes (combination of options like color/size, price). It distinguishes itself from sibling tools by emphasizing 'new'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for adding variants to products that already exist, which gives some context. However, it does not explicitly state when to use this tool versus alternatives like update_variant_price or create_product, nor does it provide exclusions or prerequisites beyond 'existing product'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• upload_image_from_urlA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden for behavioral disclosure. It only says the tool 'handles the transfer of image data to Shopify's servers', which is vague. It does not disclose whether authentication is required, what side effects may occur, how failures (like an invalid URL) are handled, or what the response contains. This is a significant gap for a write operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loads the core purpose in the first sentence. The second sentence, however, adds little value with the phrase 'It handles the transfer of image data to Shopify's servers', which is redundant with the first sentence. Overall it is concise but has one redundant clause that prevents a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, but with no output schema, the description should at least hint at the return value (e.g., uploaded image URL or ID) or constraints like supported image formats. It only mentions alt text and filename, leaving an agent uncertain about how to verify success or use the result. The lack of annotation support makes this incompleteness more damaging.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so every parameter (`image_url`, `alt_text`, `filename`) is already documented in the schema. The description adds minor context by confirming the image comes from a public URL and that the filename is optional, but it does not significantly enhance parameter meaning beyond the schema baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action 'uploads an image' and the target resource 'Shopify CDN Files section', making the tool's purpose unambiguous. It also mentions the ability to set alt text and filename, which distinguishes it from any other image-related operation. No sibling tool overlaps with this upload functionality.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying this is for uploading from a public URL to Shopify CDN Files. It does not explicitly name alternatives or list when-not-to-use conditions, but the lack of any similar sibling tools makes the intended use clear. The phrase 'public URL' also sets a prerequisite that tells the agent when this tool is applicable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• introspect_admin_schemaA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it does explain the tool's basic behavior: it introspects and returns schema sections, filtered by query and filter. It does not disclose output structure, size limits, or whether the operation is strictly read-only, though 'introspects' strongly implies a safe read action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded with the core purpose before listing arguments. It avoids excessive detail, though the second sentence partly duplicates schema information and the phrase 'relevant to the user prompt' adds unnecessary ambiguity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple, has only two well-documented parameters, and no output schema, so the description gives enough to start a call. However, because there is no output schema and no annotations, the description would be stronger if it clarified what the returned schema data looks like and how to interpret the filter options in practice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the description largely restates what the input schema already documents: query filters schema elements by name, and filter selects sections. It adds no meaningful parameter semantics beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as introspecting and returning a portion of the Shopify Admin API GraphQL schema, which is a specific verb+resource. However, it mentions returning schema 'relevant to the user prompt' even though the actual parameters are query and filter, introducing slight ambiguity about how the prompt maps to the tool call.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly restricts usage to the Shopify Admin API and warns against using it for Storefront or Functions APIs, which provides clear when/not-to-use guidance. It does not, however, contrast this tool with sibling alternatives such as query_shopify, so the differentiation from tools that execute GraphQL queries is implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_inventory_quantityA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It does reveal that the tool mutates state ('updates the available quantity') and states the return value ('returning the updated inventory level after adjustment'), but it does not mention permissions, validation rules, reversibility, or error behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficiently structured sentence that front-loads the action ('updates') and directly specifies the resource, inputs, and output. There is no filler or redundant repetition of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with three scalar parameters, no nested objects, and no output schema, the description covers the core purpose, inputs, and return value ('updated inventory level'). It is adequate for selecting and calling the tool, but it omits details about error conditions or permission requirements, though those are not critical for this simple update.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so the schema already fully explains all three parameters. The description only lists the parameter names without adding further semantic detail, such as constraints or relationships, beyond what the schema states. A baseline of 3 is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'updates the available quantity of an inventory item at a specific location,' and it lists all three inputs and the output. It is clear and unambiguous, but it does not explicitly differentiate from sibling tools such as update_variant_price or add_product_variant.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool by naming the operation, but it gives no explicit guidance about when to choose it over sibling tools or any exclusions. There is no mention of alternatives or non-use cases, leaving the agent to infer usage from the stated purpose.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• browse_ordersA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the behavioral burden. It accurately signals read-only retrieval and highlights that order_id returns detailed single-order information. However, it does not disclose pagination, default limit behavior, list-mode return shape, or how filters interact.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences front-load the core purpose, then compactly list filtering capabilities and the special order_id behavior. There is no meaningful filler; 'powerful filtering options' is slightly promotional but does not harm clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 11 parameters and no annotations or output schema, the description provides a useful overview but is incomplete. It does not explain list-mode return structure, pagination, default limit, sort behavior, or whether filters combine additively, so an agent would still need to infer several execution details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 11 parameters; the baseline is 3. The description adds slight value by grouping filters and connecting order_id to detailed output, but it mostly restates schema information rather than deepening it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The first sentence uses a specific verb ('retrieves') and resource ('Shopify orders'), clearly distinguishing this from sibling tools like browse_products and browse_customers. The final sentence adds a distinct single-order mode via order_id, making the tool's dual behavior explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for order retrieval and filtering, and it enumerates filter categories, but it does not explicitly say when to choose this over query_shopify, browse_customers, or other siblings. There are no exclusions or alternative routing cues beyond the word 'orders'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• browse_productsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses that the tool retrieves/reads data, supports pagination, and returns product details, but it does not explain how pagination is controlled given only a limit parameter, nor does it describe response format or failure behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences with no filler or repetition. It front-loads the core purpose, then adds pagination and parameter-specific behavior efficiently.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is adequate for a simple browse/read tool, especially with fully described schema parameters. However, there is no output schema, and the description does not clarify pagination mechanics or response structure, leaving a noticeable gap for an agent trying to use all features effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers all three parameters with descriptions, so the baseline is 3. The description adds minimal value: product_id is said to return detailed info for a single product, but limit and include_inventory are not meaningfully expanded beyond their schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a list of products and can optionally retrieve a single product by ID. This is a specific verb+resource combination that is distinct from siblings like browse_orders, browse_customers, and create_product.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use it to browse products, and use product_id to get one product's detail. However, it does not explicitly differentiate from sibling tools such as list_inventory_items, which is relevant because inventory information is optional here but inventory-specific tools exist.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_inventory_itemsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The verb 'retrieves' signals a read-only operation, and it adds useful behavioral context: optional filtering, pagination, and the kinds of inventory details returned. It does not mention ordering, end-of-pagination behavior, or explicitly state that no mutation occurs, but it is adequate for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no waste: the first sentence front-loads the main action and filtering capability, and the second adds pagination and returned detail types. It does not repeat parameter names or schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only list tool with three optional parameters and no output schema, the description provides the essential context: what it retrieves, filtering, pagination, and output content. It does not describe the exact response shape (since none is provided) or explicitly state the default/max limit values, which are in the schema. Overall, it is complete enough for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions SKU filtering and pagination, which loosely correspond to sku_filter, limit, and cursor, but it adds no meaning beyond the schema's own descriptions. The schema already documents defaults, max values, and filter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'retrieves a list of inventory items' with optional SKU filtering and pagination, and it clarifies what data is returned (quantities, costs, locations). The plural 'list' and its scope distinguish it from the sibling get_inventory_item, even though it does not name the alternative explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It clearly implies when to use the tool: when you need a list of inventory items, possibly filtered by SKU, with pagination support. However, it does not explicitly contrast with alternatives such as get_inventory_item for a single item or browse_products for product browsing, leaving the selection to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_variant_priceA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure. It transparently explains the mutation semantics, including null/empty-string removal of compare-at price, but doesn't mention side effects, persistence, permissions, or return behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences front-load the core operation and then describe the three supported price actions without filler. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple three-parameter mutation with full schema coverage, the description plus schema provides enough information to call the tool correctly. It lacks only minor guidance on return/error behavior, and there is no output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already fully documented in the schema. The description mostly restates the schema's semantics (e.g., null/empty removes compare-at) without adding new meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it 'updates the price and/or compare-at price of a specific product variant,' with explicit sub-actions (set price, add/update compare-at, remove compare-at). This separates it from sibling tools like add_product_variant or update_inventory_quantity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage whenever a variant's price fields need to change, but it doesn't explicitly state when not to use it or name a preferred alternative. No exclusion guidance is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_abandonmentA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the burden of disclosing behavior. It does so by stating that the tool returns items, prices, customer info, and abandonment timing, and by noting that the fields returned can be customized. It does not mention error handling or missing-ID behavior, but it clearly communicates the read-only, data-returning nature of the operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. It leads with the core purpose, then lists the returned data categories, then mentions the customization option. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple by-ID retrieval tool, the description covers the main purpose and return content well. However, the fields parameter is not explained in terms of acceptable values, and with no output schema, the agent does not know the exact default response structure or how to request specific fields beyond the vague 'customize' wording.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds some meaning by confirming that the 'fields' parameter controls which fields are returned, but it does not enumerate possible field names or explain how to structure the fields object, which remains opaque due to additionalProperties being true.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('retrieves') and names the exact resource ('cart or checkout abandonments') with a clear lookup mechanism (by ID). It clearly distinguishes itself from the sibling tools, none of which target abandonment records.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool should be used when you have an abandonment ID and need its details, but it does not explicitly state when to prefer this tool over alternatives or call out any other tool. No exclusions or alternative routing are provided, leaving usage somewhat implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_shopifyA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry behavioral disclosure. It does mention mutation capability and the need to 'enable mutation operations', which signals potential write effects. However, it does not elaborate on consequences of mutations (e.g., irreversibility, permission requirements, rate limits) or error behavior. Adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero fluff. The core purpose and distinguishing scope are front-loaded, and the 'options' clause covers the additional parameters efficiently. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a flexible catch-all with three well-documented parameters and no output schema, the description is mostly complete. It covers purpose, scope, and the mutation gate. A minor gap is that it doesn't mention how to discover available endpoints, though the sibling `introspect_admin_schema` tool exists for that purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all three parameters. The description adds marginal interpretive value by framing `query` as custom GraphQL and mentioning that variables and mutation-enabling are options, but it does not provide syntax, formatting, or deeper semantic guidance beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('executes'), a clear resource ('custom GraphQL queries or mutations against the Shopify Admin API'), and differentiates the tool from siblings by framing it as a flexible fallback for endpoints 'not covered by the specialized tools'. This is immediately distinguishable from the specialized sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use this tool: when you need an endpoint 'not covered by the specialized tools'. It does not list specific sibling tools or give explicit 'do not use when' examples, but the condition is clear enough for an agent to route appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_productA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full behavioral disclosure. It clearly indicates that the product is created in DRAFT status, handles a defined set of fields, and explicitly states that variants with prices are not created. This is meaningful behavioral context beyond what the parameter names alone would convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence covers the core purpose and status; the second adds the critical limitation and usage guidance. Every part contributes to the agent's understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, status, and boundary conditions well, and parameter coverage is complete. However, there is no output schema and the description does not mention what the tool returns (e.g., the created product ID), which is important for 'first step' workflows followed by variant creation. That gap lowers completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description adds little parameter-level meaning beyond grouping them as 'product title, description, vendor, type, tags, and options configuration,' which duplicates the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('creates') and resource ('product structure with options'), specifies the DRAFT status, and explicitly differentiates itself from variant creation by noting it 'does not create variants with prices.' This allows an agent to distinguish it from sibling tools like add_product_variant without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear usage context: 'Use this as the first step in product creation.' It also defines an implicit boundary—no variants or prices—which helps agent know when not to use this tool in favor of add_product_variant. However, it does not explicitly name the alternative tool, so an agent must infer that add_product_variant is the follow-up.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_inventory_itemA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It clearly characterizes the operation as a read/retrieval and adds useful behavioral detail about scope, such as 'available quantities across all locations.' It does not discuss errors or permissions, but this is a low-risk read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: it states the action, the lookup keys, and the kind of data returned in two sentences. Every clause contributes useful information with minimal filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with no output schema and no annotations, the description covers purpose, lookup parameters, and return contents well. The main gap is that the need for at least one of id/sku is left implicit despite the schema marking both as optional, but the core invocation path is still clear.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Both parameters are already fully described in the schema with examples, so schema_description_coverage is 100%. The description adds little beyond echoing 'by ID or SKU' and does not clarify whether at least one is required or what happens if both are supplied, but it is not required to compensate for schema gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'retrieves detailed information about a specific inventory item by ID or SKU.' It also enumerates return contents, distinguishing this singular lookup from sibling tools like list_inventory_items.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The tool's context is clearly implied: use it when you have an ID or SKU and need detailed single-item data. It does not explicitly name alternatives or exclusions, but the 'specific item' wording makes the intended selection reasonably obvious.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.