Server Quality Checklist

Tool Scores

• shopify_list_publicationsC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish readOnlyHint, idempotentHint, and non-destructive behavior, so the description does not need to repeat safety traits. It adds a small amount of context by specifying that only publish-available publications are returned, but it does not disclose pagination or response behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one short, front-loaded sentence with no filler. For a simple list operation with one optional parameter, this is appropriately concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema and the description does not explain what fields are returned (e.g., publication IDs needed by publish_product) or how the 'first' parameter behaves. The read-only annotations reduce safety concerns, but the lack of parameter semantics and return-value context leaves the definition incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention the only parameter, 'first'. An agent is left to infer that 'first' is a page-size/limit control solely from the integer type, minimum, maximum, and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('列出' list) and resource ('Shopify publications') with a scope qualifier ('可用于发布商品', available for publishing products). It clearly distinguishes from product-focused siblings like shopify_search_products, though it does not explicitly name an alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to choose this tool over siblings, nor when not to use it. The phrase 'available for publishing products' implies a pre-publish step, but there is no explicit context or mention of alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_search_productsB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, idempotent, and non-destructive behavior, lowering the burden on the description. The description adds that results are paginated but does not explain cursor semantics, result shape, or iteration behavior with the `after` parameter.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single compact sentence states the operation, the query grammar, and the pagination behavior with no filler. It is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and read-only, and the description covers the basic operation, but it leaves out how pagination is driven by `after`, and with no output schema it does not describe what the returned product objects contain. These are meaningful gaps for an agent, though not fatal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, so the description must compensate. It clarifies that `query` uses Shopify search syntax, but it does not explain `after` (pagination cursor) or `first` (page size) beyond what the schema already gives.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('query') and resource ('products'), and adds that it uses Shopify search syntax and returns paginated results. It clearly signals a search/list operation, but it does not explicitly contrast itself with shopify_get_product for single-product lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to prefer this tool over sibling tools, such as using shopify_get_product for a known product ID. The intended use is only implied by the search phrasing and pagination mention, not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_create_product_draftB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The annotations (all false hints) provide minimal behavioral signal, so the description supplies the important fact that this endpoint never publishes or lists the product. It does not discuss idempotency, validation failures, or the shape of the created draft, but the non-publication constraint is a meaningful disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One tight sentence with the action front-loaded and the critical negative behavior appended as a second clause. There is no wasted wording or redundant restatement of the title.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a create operation with seven parameters, no output schema, and no significant annotations, the description covers only the draft/non-publish behavior. It omits expected return value/confirmation and how to later publish the draft, leaving meaningful gaps for an agent choosing and invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no parameter-level meaning. The property names (title, handle, vendor, productType, descriptionHtml, tags, seo) are self-explanatory, but the description does not compensate for the missing schema descriptions or clarify relationships between parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description identifies a concrete verb ('创建') and resource ('DRAFT 商品'), and adds the key scope that the tool will not auto-publish/list to sales channels. This clearly separates it from publish/status tools, though it could name the lifecycle position more explicitly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The '不会自动上架或发布' clause implies the tool is for products not yet ready for sale, which gives some usage context. However, it does not state when to prefer this over update_product or set_product_status, nor does it point to publish_product as the intended follow-up to make a draft live.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_connection_statusA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark the tool as read-only, idempotent, and non-destructive. The description adds meaningful behavioral context by stating it verifies connection, API version, and authorization scopes, and explicitly discloses that no secrets/keys are returned, which is valuable beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that states the core purpose and a key behavioral guarantee. Every word earns its place, with no redundant or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only status check tool, the description is mostly complete: it names what is verified and explicitly says no secrets are returned. However, it does not describe the exact output format or fields returned, which would have made it fully self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and schema description coverage is 100%, so the parameter dimension is trivially satisfied. The baseline for zero-parameter tools is 4, and no additional parameter explanation is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: verify Shopify connection, API version, and authorization scopes. It is specific about the resource and scope, though it does not explicitly distinguish itself from the similar sibling shopify_config_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. It does not mention prerequisites, typical call scenarios, or conditions under which another sibling should be used instead.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_unpublish_productA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  标注已声明 destructiveHint=true，描述进一步澄清破坏范围：只会从销售渠道撤下，不会删除商品，并补充“调用前应获得用户确认”的操作要求。这超越了标注本身提供的信息，且与标注不矛盾。

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  描述只有两句，行为、范围、关键边界和前置确认要求全部前置呈现，无冗余信息。每个句子都承担了必要的信息量。

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  对于仅有两个参数、无嵌套对象、无输出 schema 的简单工具，描述配合已有标注和 schema 已足以支撑正确调用。缺少返回值或错误行为说明，但这不是调用成功的关键信息。

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  schema 描述覆盖率为 0%，但描述中“商品”和“指定销售渠道”能为 id 和 publicationIds 提供基本语义映射。不过描述未明确说明 id 是 Product GID、publicationIds 是 Publication GID 列表，也未解释约束条件，因此只部分弥补了 schema 的不足。

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  描述使用明确动词“撤下”和资源“商品/销售渠道”，清楚说明该工具从指定渠道下架商品而非删除商品。虽未点名对比兄弟工具，但“不会删除商品”与 publish_product/set_product_status 等在语义上有区分。

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  描述隐含了使用场景：当需要从特定销售渠道撤下商品时使用，并通过“不会删除商品”排除了删除用途。但未明确说明与 shopify_publish_product、shopify_set_product_status 的选型关系，也未给出何时不应使用本工具。

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_get_productA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is covered. The description adds a non-obvious behavioral limit—only the first 20 variants are returned—which is useful beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence with no filler or repetition. Every element—reading a product and the variant limit—earns its place and is easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-by-id tool, the description provides the key operational fact: it returns one product and at most 20 variants. The required parameter format is fully specified in the schema, and annotations cover safety. It does not describe return fields, but no output schema exists and the returned resource is clearly named.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is only one parameter, `id`, and schema description coverage is 0%. The description does not explain that the id must be a Shopify product GID or otherwise add meaning beyond the field name and pattern. The schema's pattern helps, but the description fails to compensate for the lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a concrete read operation on a specific resource ('读取一个商品') and adds a precise scope ('前 20 个变体'). This clearly distinguishes it from write tools like shopify_update_product and from search tools like shopify_search_products.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a single product and its first variants are needed, but it gives no explicit when-not-to-use guidance or mention of alternatives such as shopify_search_products for finding products. The intended use case is reasonably inferable, not explicitly routed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_publish_productA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate the operation is not read-only, but the description adds meaningful behavioral context by stating that store visibility changes and that user confirmation should be obtained before calling. This goes beyond the annotation metadata and helps the agent understand side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no redundancy. The core purpose is front-loaded, followed by the key behavioral caveat and confirmation requirement. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderate-complexity mutation with no output schema, the description plus schema and annotations is sufficient for correct invocation: it states the product prerequisite (ACTIVE), the target (sales channels), the effect (visibility change), and the need for confirmation. Minor omissions like return values or error behavior are not critical for selecting and calling this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 33%, covering publishAt. The description partially compensates by indicating that id must refer to an ACTIVE product and that publicationIds are the target sales channels, but it does not provide detailed parameter-level semantics beyond what the schema already offers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: publishing ACTIVE products to specified sales channels. This specific verb+resource combination distinguishes it from sibling tools like shopify_unpublish_product or shopify_set_product_status, even without naming them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool—when an ACTIVE product should be made visible on a sales channel—and includes a confirmation requirement. However, it does not explicitly mention alternatives or state when not to use it versus other siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_set_product_statusA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly=false, idempotent=true, destructive=false. The description goes beyond them by warning that setting ACTIVE does not publish the product, which is a non-obvious behavioral trait. This is useful context an agent would otherwise miss from the schema and annotations alone.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that wastes no words. It states the operation, lists all valid statuses, and includes the most important caveat in one breath.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter, non-destructive, idempotent status setter, the description carries the main operational caveat (ACTIVE ≠ published) and the sibling list supplies adjacent tools. It does not explain response/return behavior, but no output schema exists and the operation is simple enough that this is not a critical gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the schema itself has a clear enum for status and a GID pattern for id. The description adds the meaningful distinction that ACTIVE is not a publication state, yet it gives no additional guidance for the id parameter or for the implications of the other statuses.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('把商品设为') on a specific resource ('商品'), enumerates the four allowed statuses, and adds the clarifying caveat that ACTIVE is not the same as published. This makes the tool's purpose unmistakable and separates it from publication-related siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The caveat 'ACTIVE 不等于已发布' implies this tool should not be used for publication workflows, but it never explicitly says to use shopify_publish_product/shopify_unpublish_product for that. The guidance is therefore implied rather than stated, which leaves routing to an inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_update_productA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate a write operation (readOnlyHint=false), idempotency, and non-destructiveness. The description adds a behavioral guarantee not captured in annotations: it will not alter publication channels. This is meaningful context for tool selection. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single concise sentence with the action front-loaded and the key constraint following. Every word carries meaning, no redundancy, and it is immediately scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple update operation, the description plus annotations provide a usable picture: idempotent, non-destructive, and preserving publication channels. However, with 8 parameters and 0% schema descriptions, and no output schema, the description leaves parameter semantics and return behavior to inference, so it is not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter documentation. It only groups fields under '基础信息' without explaining individual parameters, value formats, or relationships. The schema provides property names and types, but the agent is left to infer semantics from names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific action ('update') on a specific resource ('product') and adds a scope constraint ('basic info'), which clearly separates it from publish/unpublish siblings. The exclusion '不改变发布渠道' further sharpens the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says it does not change publication channels, which tells the agent when NOT to use it relative to publish/unpublish tools. However, it does not name the alternative sibling tools for channel changes, so the guidance is clear but not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• shopify_config_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds useful behavioral context beyond annotations: it only reports existence, does not test network connectivity, and deliberately avoids returning secrets. This gives the agent an accurate operational expectation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly scoped sentence that front-loads the core behavior and then adds two important exclusions. Every clause earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only status tool with annotations covering safety, the description is complete. It states what is returned, what is not returned, and what is not tested, which is enough for an agent to select and invoke the tool correctly without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so parameter semantics are trivially complete. The description confirms the call requires no configuration inputs and only returns status information, which is sufficient for a no-argument tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource: it returns only whether configuration source and credentials exist. It also explicitly states what it does not do (test network connection, return secrets), clearly distinguishing it from sibling shopify_connection_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly indicates when to use this tool: when checking config/credential existence without network testing. It does not name the alternative tool explicitly, but the exclusion of network testing implies shopify_connection_status as the sibling alternative, so guidance is strong though not fully explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.