Server Quality Checklist

Tool Scores

• get_shopB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden of disclosing behavioral traits. It simply states 'Return basic information' without mentioning any side effects, permissions, rate limits, or potential gotchas. Even though the tool is a simple read operation, the description does not explicitly confirm that it is read-only or disclose any operational behavior beyond the action itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that immediately conveys the tool's purpose. There is no fluff or redundant information. Every word contributes to understanding what the tool does, making it an excellent example of concise, front-loaded communication.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, output schema exists), the description is reasonably complete. The output schema likely details the return structure, so the description does not need to enumerate fields. However, it could have added a bit more context about what 'basic information' entails or any prerequisites, but for a straightforward getter, this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4 as per the rubric. The description adds no parameter details because none exist. The schema is automatically fully covered since the property list is empty. The description's mention of 'basic information' aligns well with the tool's function and requires no further parameter explanation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear action and resource: 'Return basic information about the connected Shopify shop.' It is distinct from the sibling tools (list_products, get_product, list_orders) which target products and orders, so there is no ambiguity about what this tool does. The phrase 'basic information' is somewhat generic but does not detract from the clarity of the overall purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention any conditions, exclusions, or scenarios where another tool might be more appropriate. For example, it does not say 'use this to retrieve shop-level settings' or contrast it with product/order tools. The only implicit guidance is that it is for shop info, but no explicit 'when to use' is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ordersB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It only states the action 'list' and the recency of orders, but does not mention pagination, rate limits, required permissions, or any side effects. The behavior is mostly implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that includes the core purpose and a key usage detail. It is front-loaded with the verb and resource, with no unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is relatively simple with an output schema present, so return values are available. However, the description omits details about pagination parameters and behavioral expectations, leaving gaps for a complete understanding. Adequate but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It adds meaning for 'search' by referencing Shopify order search syntax, but does not explain 'after' or 'limit', leaving these parameters unclear. Partial compensation only.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List recent orders', specifying the exact action (list) and resource (orders). It distinguishes from sibling tools like list_products and get_product by targeting orders specifically.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for listing orders, and the mention of optional Shopify search syntax suggests when to use the search parameter. However, it does not explicitly address when not to use this tool or provide alternative context, so guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_productsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. 'List' implies a read-only operation, and 'Shopify product search syntax' hints at search behavior, but the description does not disclose pagination details, ordering, or any side effects. It is minimally transparent but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that conveys the core purpose and adds a relevant usage detail. Every word earns its place, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with an output schema, the description covers the primary purpose but lacks explicit parameter guidance and behavioral context like pagination. It is adequate but has clear gaps, particularly around the 'after' and 'limit' parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds some meaning for the 'search' parameter by mentioning 'Shopify product search syntax', but it says nothing about 'after' (likely pagination cursor) or 'limit' (page size). The parameter semantics are only partially explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List products, optionally using Shopify product search syntax' clearly states the verb ('List') and resource ('products'), and the reference to search syntax distinguishes it from the sibling tool get_product which retrieves a single product. It is specific and immediately understandable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is for listing multiple products rather than fetching a single one, which differentiates it from get_product and fits the sibling set. It does not explicitly name alternatives or exclusions, but the context is unambiguous enough for an agent to select it appropriately.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_productA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It communicates the lookup mechanism (by numeric ID or GID) and the read-only nature via the verb 'Get', but it lacks disclosure about error handling, permissions, or return behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no redundant words. Every word adds value, covering both purpose and parameter meaning.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter with one parameter and an output schema present, the description adequately covers what the tool does and the acceptable ID formats. It is slightly less complete because it omits any guidance on not-found behavior or edge cases, which would be useful but not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only shows a string parameter named 'product_id' with no description. The description compensates by explaining that the ID can be 'numeric ID or Shopify GID', giving essential context about expected parameter formats beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' with the resource 'product' and explicitly clarifies the accepted identifier formats ('numeric ID or Shopify GID'), making it distinct from the sibling tool list_products.This directly states what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is used to retrieve a single product when you have a specific ID, and its phrasing contrasts with list_products. However, it does not explicitly mention alternatives or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.